2023-03-01T19:29:40.511 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8 2023-03-01T19:29:40.512 INFO:teuthology.results:subset: '111/120000' 2023-03-01T19:29:40.513 INFO:teuthology.results:seed: '3767' 2023-03-01T22:01:41.225 INFO:root:teuthology version: 0.0.1.dev49+gc7c3cc8 2023-03-01T22:01:41.255 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-03-02T00:30:56.308 INFO:teuthology.results:Tests finished! gathering results... 2023-03-02T00:31:20.223 INFO:scrape:Found 317 jobs 2023-03-02T00:31:21.631 INFO:scrape:Found 7 distinct failure reasons 2023-03-02T00:31:21.632 INFO:scrape:Failure: Command failed on smithi116 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:f3c53b1df0bab0fdd94bf5083cc8f394b0a03d35 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b32f6170-b86c-11ed-9ae9-001a4aab830c -- ceph mon dump -f json' 2023-03-02T00:31:21.632 INFO:scrape:1 jobs: ['7190289'] 2023-03-02T00:31:21.633 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rmdir-reactivate}', 'rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8'] 2023-03-02T00:31:21.633 INFO:scrape: 2023-03-02T00:31:21.633 INFO:scrape:Failure: Command failed (workunit test cephadm/test_cephadm.sh) on smithi123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f3c53b1df0bab0fdd94bf5083cc8f394b0a03d35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' 2023-03-02T00:31:21.634 INFO:scrape:2 jobs: ['7190502', '7190371'] 2023-03-02T00:31:21.634 INFO:scrape:suites intersection: ['agent/on', 'mon_election/connectivity', 'task/test_cephadm}'] 2023-03-02T00:31:21.634 INFO:scrape:suites union: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_3.0', 'rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_rhel8', 'task/test_cephadm}'] 2023-03-02T00:31:21.634 INFO:scrape: 2023-03-02T00:31:21.635 INFO:scrape:Failure: Command failed on smithi102 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' 2023-03-02T00:31:21.635 INFO:scrape:7 jobs: ['7190522', '7190444', '7190402', '7190365', '7190434', '7190284', '7190488'] 2023-03-02T00:31:21.635 INFO:scrape:suites intersection: [] 2023-03-02T00:31:21.636 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-pg-log-overrides/normal_pg_log', '1-rook', '2-recovery-overrides/{default}', '2-workload/none', '2-workload/radosbench', '3-final', '3-scrub-overrides/{default}', 'backoff/normal', 'ceph', 'cluster/1-node', 'cluster/3-node', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'k8s/1.21', 'mon_election/classic', 'mon_election/connectivity', 'msgr-failures/many', 'msgr-failures/osd-dispatch-delay', 'msgr/async-v1only', 'msgr/async-v2only', 'net/calico', 'net/flannel', 'net/host', 'no_pools', 'objectstore/bluestore-bitmap', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-stupid', 'openstack}', 'rados', 'rados/basic/{ceph', 'rados/multimon/{clusters/21', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'rook/1.7.2}', 'rook/master}', 'supported-random-distro$/{rhel_8}', 'supported-random-distro$/{ubuntu_latest}', 'tasks/mon_clock_no_skews}', 'tasks/rados_striper}', 'thrashers/careful', 'thrashosds-health', 'workloads/small-objects}'] 2023-03-02T00:31:21.636 INFO:scrape: 2023-03-02T00:31:21.637 INFO:scrape:Failure: Cannot connect to remote host smithi050 2023-03-02T00:31:21.637 INFO:scrape:1 jobs: ['7190403'] 2023-03-02T00:31:21.637 INFO:scrape:suites: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/pacific', 'backoff/peering_and_degraded', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'thrashers/careful', 'thrashosds-health', 'three-plus-one}', 'workloads/cache-snaps}'] 2023-03-02T00:31:21.638 INFO:scrape: 2023-03-02T00:31:21.638 INFO:scrape:Failure: {'smithi029.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'only_upgrade': False, 'package': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'policy_rc_d': None, 'purge': False, 'state': 'absent', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': "'apt-get remove 'librbd1' 'librados2'' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14072 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14072 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n', 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14072 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}} 2023-03-02T00:31:21.638 INFO:scrape:1 jobs: ['7190354'] 2023-03-02T00:31:21.638 INFO:scrape:suites: ['mon_election/classic', 'rados', 'rados/singleton-nomsgr/{all/version-number-sanity', 'supported-random-distro$/{ubuntu_latest}}'] 2023-03-02T00:31:21.639 INFO:scrape: 2023-03-02T00:31:21.639 INFO:scrape:Failure: Command failed (workunit test scrub/osd-recovery-scrub.sh) on smithi110 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f3c53b1df0bab0fdd94bf5083cc8f394b0a03d35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery-scrub.sh' 2023-03-02T00:31:21.639 INFO:scrape:1 jobs: ['7190326'] 2023-03-02T00:31:21.640 INFO:scrape:suites: ['rados/standalone/{supported-random-distro$/{ubuntu_latest}', 'workloads/scrub}'] 2023-03-02T00:31:21.640 INFO:scrape: 2023-03-02T00:31:21.640 INFO:scrape:Failure: SSH connection to smithi050 was lost: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' 2023-03-02T00:31:21.641 INFO:scrape:1 jobs: ['7190417'] 2023-03-02T00:31:21.641 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_nfs}'] 2023-03-02T00:31:21.641 INFO:scrape: 2023-03-02T00:31:21.655 INFO:teuthology.results:Sending results to yweinstein@radhat.com: Test Run: yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/ failed: 12 dead: 2 running: 0 waiting: 0 queued: 0 passed: 303 Fail ================================================================= [7190284] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/host rook/master} ----------------------------------------------------------------- time: 00:06:43 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190284/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190284/ sentry: https://sentry.ceph.com/organizations/ceph/?query=29dd2c6fa8e4408b8ec29a435c80d7e0 Command failed on smithi175 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7190289] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} ----------------------------------------------------------------- time: 00:12:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190289/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190289/ sentry: https://sentry.ceph.com/organizations/ceph/?query=885240abd1cf4816a481f1fb634bffa7 Command failed on smithi116 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:f3c53b1df0bab0fdd94bf5083cc8f394b0a03d35 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid b32f6170-b86c-11ed-9ae9-001a4aab830c -- ceph mon dump -f json' [7190326] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/scrub} ----------------------------------------------------------------- time: 00:14:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190326/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190326/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bf9c5064f20e4b0d88a5edc2bf5a5f34 Command failed (workunit test scrub/osd-recovery-scrub.sh) on smithi110 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f3c53b1df0bab0fdd94bf5083cc8f394b0a03d35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/standalone/scrub/osd-recovery- scrub.sh' [7190365] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:06:01 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190365/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190365/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7ee42296881c4c6f8f645e10e5cc866c Command failed on smithi170 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7190371] rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:13:43 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190371/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190371/ sentry: https://sentry.ceph.com/organizations/ceph/?query=febfaf5cda9641848f27cba864083af7 Command failed (workunit test cephadm/test_cephadm.sh) on smithi057 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f3c53b1df0bab0fdd94bf5083cc8f394b0a03d35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7190402] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/small-objects} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190402/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190402/ sentry: https://sentry.ceph.com/organizations/ceph/?query=8ba6edf9804e4a7ea5a64fa05c37275d Command failed on smithi050 with status 1: 'sudo yum install -y kernel' [7190403] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/careful thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190403/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190403/ sentry: https://sentry.ceph.com/organizations/ceph/?query=eff1601944b845d4967c883bf587b8c4 Cannot connect to remote host smithi050 [7190434] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} tasks/rados_striper} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190434/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190434/ sentry: https://sentry.ceph.com/organizations/ceph/?query=9f58954dd69146eea8f131d12c763691 Command failed on smithi050 with status 100: 'sudo apt-get clean' [7190444] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none 3-final cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:06:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190444/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190444/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6777932697da410fa8c804f66371ae0b Command failed on smithi003 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' [7190488] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_no_skews} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190488/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190488/ sentry: https://sentry.ceph.com/organizations/ceph/?query=986b1ec54f354ad5add790fe0648f656 Command failed on smithi187 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' [7190502] rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:12:49 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190502/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190502/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a72f724311894914b6c83c0fdcd5771a Command failed (workunit test cephadm/test_cephadm.sh) on smithi123 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f3c53b1df0bab0fdd94bf5083cc8f394b0a03d35 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_cephadm.sh' [7190522] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench 3-final cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:06:02 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190522/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190522/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2b4f1c15a41e496b94f230b0d2026966 Command failed on smithi102 with status 1: 'sudo systemctl enable --now kubelet && sudo kubeadm config images pull' Dead ================================================================= [7190354] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:03:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190354/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190354/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2d7afb84e72a467996fbc3a95bd63a3f {'smithi029.front.sepia.ceph.com': {'_ansible_no_log': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force- confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python- ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi-common'], 'only_upgrade': False, 'package': ['ceph', 'ceph-common', 'libcephfs1', 'radosgw', 'python-ceph', 'python-rados', 'python-cephfs', 'python-rbd', 'librbd1', 'librados2', 'ceph-fs-common-dbg', 'ceph-fs-common', 'openmpi- common'], 'policy_rc_d': None, 'purge': False, 'state': 'absent', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': "'apt-get remove 'librbd1' 'librados2'' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14072 (apt-get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?\n", 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14072 (apt- get)\nE: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock- frontend), is another process using it?\n', 'stderr_lines': ['W: --force- yes is deprecated, use one of the options starting with --allow instead.', 'E: Could not get lock /var/lib/dpkg/lock-frontend. It is held by process 14072 (apt-get)', 'E: Unable to acquire the dpkg frontend lock (/var/lib/dpkg/lock-frontend), is another process using it?'], 'stdout': '', 'stdout_lines': []}} [7190417] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190417/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190417/ sentry: https://sentry.ceph.com/organizations/ceph/?query=acd2e16cb9fd431d803fbfdb0ac2a128 SSH connection to smithi050 was lost: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic' Pass ================================================================= [7190256] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190256/ [7190257] rados/standalone/{supported-random-distro$/{centos_8} workloads/osd-backfill} time: 03:27:03 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190257/ [7190258] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:22:51 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190258/ [7190259] rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_3.0 agent/off mon_election/classic task/test_cephadm_repos} time: 00:09:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190259/ [7190260] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:34:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190260/ [7190261] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:25:21 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190261/ [7190262] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{rhel_8} tasks/progress} time: 00:30:59 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190262/ [7190263] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/repair_test} time: 00:18:56 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190263/ [7190264] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/rbd_cls} time: 00:23:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190264/ [7190265] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:10:25 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190265/ [7190266] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190266/ [7190267] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:15:24 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190267/ [7190268] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:11:37 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190268/ [7190269] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:29 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190269/ [7190270] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:33:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190270/ [7190271] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/one workloads/rados_5925} time: 00:14:49 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190271/ [7190272] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:24 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190272/ [7190273] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/rados_api_tests validater/valgrind} time: 00:54:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190273/ [7190274] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:14:15 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190274/ [7190275] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:14:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190275/ [7190276] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:24:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190276/ [7190277] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{rhel_8}} time: 02:15:05 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190277/ [7190278] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:18:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190278/ [7190279] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:31:56 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190279/ [7190280] rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_rhel8 agent/on mon_election/connectivity task/test_nfs} time: 00:31:50 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190280/ [7190281] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:33:22 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190281/ [7190282] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:13:59 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190282/ [7190283] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-bitmap} tasks/e2e} time: 00:24:57 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190283/ [7190285] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190285/ [7190286] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} time: 00:19:47 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190286/ [7190287] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:30:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190287/ [7190288] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:11:10 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190288/ [7190290] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:14:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190290/ [7190291] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:15:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190291/ [7190292] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:32 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190292/ [7190293] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:31:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190293/ [7190294] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:16:14 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190294/ [7190295] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190295/ [7190296] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/osd} time: 02:55:37 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190296/ [7190297] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:16:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190297/ [7190298] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/classic random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{rhel_8} tasks/prometheus} time: 00:16:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190298/ [7190299] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:17:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190299/ [7190300] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:13:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190300/ [7190301] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:17:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190301/ [7190302] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:19:24 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190302/ [7190303] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:16:22 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190303/ [7190304] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/rados_api_tests} time: 00:32:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190304/ [7190305] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/lockdep} time: 00:20:34 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190305/ [7190306] rados/singleton/{all/mon-config mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:13:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190306/ [7190307] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{centos_8}} time: 02:25:53 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190307/ [7190308] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:11:03 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190308/ [7190309] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190309/ [7190310] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:37:04 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190310/ [7190311] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli} time: 00:18:42 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190311/ [7190312] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:17:08 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190312/ [7190313] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:05 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190313/ [7190314] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:19:19 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190314/ [7190315] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190315/ [7190316] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:23:17 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190316/ [7190317] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:57 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190317/ [7190318] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:21:14 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190318/ [7190319] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:25:43 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190319/ [7190320] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:28:26 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190320/ [7190321] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:24:53 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190321/ [7190322] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190322/ [7190323] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:51 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190323/ [7190324] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:29:49 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190324/ [7190325] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:22:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190325/ [7190327] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:15:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190327/ [7190328] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_8} tasks/workunits} time: 00:12:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190328/ [7190329] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:10:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190329/ [7190330] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:24:26 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190330/ [7190331] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:26:22 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190331/ [7190332] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:27:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190332/ [7190333] rados/objectstore/{backends/objectstore-filestore-memstore supported-random-distro$/{rhel_8}} time: 00:34:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190333/ [7190334] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/sync workloads/rados_mon_osdmap_prune} time: 00:45:19 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190334/ [7190335] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/rados_api_tests} time: 00:29:54 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190335/ [7190336] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados tasks/mon_recovery validater/valgrind} time: 00:33:08 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190336/ [7190337] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:27:53 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190337/ [7190338] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:59 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190338/ [7190339] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:56 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190339/ [7190340] rados/singleton/{all/peer mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:12:44 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190340/ [7190341] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190341/ [7190342] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/radosbench} time: 01:04:37 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190342/ [7190343] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:34:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190343/ [7190344] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_adoption} time: 00:15:16 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190344/ [7190345] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:33:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190345/ [7190346] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:19:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190346/ [7190347] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:30:56 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190347/ [7190348] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/redirect} time: 00:17:40 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190348/ [7190349] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:10:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190349/ [7190350] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:24:19 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190350/ [7190351] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:22:43 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190351/ [7190352] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:18:11 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190352/ [7190353] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/filestore-xfs rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:23:08 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190353/ [7190355] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:35:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190355/ [7190356] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect_promote_tests} time: 00:16:17 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190356/ [7190357] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190357/ [7190358] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/rados_cls_all} time: 00:18:58 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190358/ [7190359] rados/multimon/{clusters/21 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:26:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190359/ [7190360] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:12:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190360/ [7190361] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-lz4} tasks/dashboard} time: 01:08:30 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190361/ [7190362] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:11:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190362/ [7190363] rados/objectstore/{backends/alloc-hint supported-random-distro$/{centos_8}} time: 00:12:30 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190363/ [7190364] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:13:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190364/ [7190366] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:18:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190366/ [7190367] rados/standalone/{supported-random-distro$/{rhel_8} workloads/c2c} time: 00:16:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190367/ [7190368] rados/upgrade/parallel/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:23:17 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190368/ [7190369] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:22:04 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190369/ [7190370] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/redirect_set_object} time: 00:17:33 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190370/ [7190372] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_workunits} time: 00:30:21 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190372/ [7190373] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/filestore-xfs rados tasks/rados_api_tests validater/lockdep} time: 00:27:29 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190373/ [7190374] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/filestore-xfs rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:48 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190374/ [7190375] rados/singleton/{all/radostool mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:25:41 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190375/ [7190376] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:57 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190376/ [7190377] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:11:24 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190377/ [7190378] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:01 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190378/ [7190379] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:08:53 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190379/ [7190380] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/set-chunks-read} time: 00:18:40 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190380/ [7190381] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:25:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190381/ [7190382] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:28:48 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190382/ [7190383] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:14:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190383/ [7190384] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:30:27 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190384/ [7190385] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:44 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190385/ [7190386] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/small-objects-balanced} time: 00:19:39 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190386/ [7190387] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:16:32 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190387/ [7190388] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:03 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190388/ [7190389] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:18:44 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190389/ [7190390] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190390/ [7190391] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:12:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190391/ [7190392] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/small-objects-localized} time: 00:23:10 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190392/ [7190393] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{centos_8}} time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190393/ [7190394] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_clock_no_skews} time: 00:12:15 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190394/ [7190395] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} time: 00:25:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190395/ [7190396] rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm_repos} time: 00:09:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190396/ [7190397] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:08:55 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190397/ [7190398] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{ubuntu_latest} tasks/failover} time: 00:13:10 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190398/ [7190399] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} time: 00:30:14 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190399/ [7190400] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:10:58 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190400/ [7190401] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:23:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190401/ [7190404] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:58:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190404/ [7190405] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:50 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190405/ [7190406] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190406/ [7190407] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:12:21 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190407/ [7190408] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:28:58 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190408/ [7190409] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects-balanced} time: 00:30:22 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190409/ [7190410] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190410/ [7190411] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} time: 00:18:01 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190411/ [7190412] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:05 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190412/ [7190413] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8}} time: 00:13:39 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190413/ [7190414] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:27:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190414/ [7190415] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/snaps-few-objects-localized} time: 00:31:16 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190415/ [7190416] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:08:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190416/ [7190418] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:08:54 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190418/ [7190419] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190419/ [7190420] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:12:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190420/ [7190421] rados/objectstore/{backends/filejournal supported-random-distro$/{centos_8}} time: 00:11:24 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190421/ [7190422] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:29:47 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190422/ [7190423] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:12:26 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190423/ [7190424] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190424/ [7190425] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:08:54 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190425/ [7190426] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:23:14 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190426/ [7190427] rados/singleton/{all/test_envlibrados_for_rocksdb mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{centos_8}} time: 00:19:50 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190427/ [7190428] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:19:56 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190428/ [7190429] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/one workloads/snaps-few-objects} time: 00:39:01 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190429/ [7190430] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:15:48 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190430/ [7190431] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/none thrashosds-health workloads/write_fadvise_dontneed} time: 00:17:03 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190431/ [7190432] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-hybrid} supported-random-distro$/{centos_8} tasks/insights} time: 00:16:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190432/ [7190433] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:13:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190433/ [7190435] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:35:04 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190435/ [7190436] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:09 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190436/ [7190437] rados/standalone/{supported-random-distro$/{rhel_8} workloads/erasure-code} time: 01:29:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190437/ [7190438] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest}} time: 01:15:50 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190438/ [7190439] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:36 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190439/ [7190440] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli} time: 00:16:59 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190440/ [7190441] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/admin_socket_objecter_requests} time: 00:46:34 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190441/ [7190442] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:09:02 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190442/ [7190443] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-bitmap} tasks/e2e} time: 00:24:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190443/ [7190445] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:30 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190445/ [7190446] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/filestore-xfs rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/fastread thrashosds-health workloads/ec-radosbench} time: 01:06:48 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190446/ [7190447] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:47:43 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190447/ [7190448] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/default thrashosds-health workloads/radosbench} time: 01:43:47 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190448/ [7190449] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:16 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190449/ [7190450] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-big} time: 00:32:55 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190450/ [7190451] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:28:26 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190451/ [7190452] rados/objectstore/{backends/filestore-idempotent-aio-journal supported-random-distro$/{ubuntu_latest}} time: 02:26:28 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190452/ [7190453] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:23:29 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190453/ [7190454] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:14:37 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190454/ [7190455] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:15:17 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190455/ [7190456] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:18:49 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190456/ [7190457] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/cache-agent-small} time: 00:16:17 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190457/ [7190458] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:30:21 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190458/ [7190459] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:21:50 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190459/ [7190460] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:26:40 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190460/ [7190461] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:12:08 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190461/ [7190462] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:31:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190462/ [7190463] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:09:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190463/ [7190464] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/sync-many workloads/pool-create-delete} time: 00:20:34 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190464/ [7190465] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:12:57 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190465/ [7190466] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/filestore-xfs rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:29:58 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190466/ [7190467] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/module_selftest} time: 00:27:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190467/ [7190468] rados/cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:16:21 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190468/ [7190469] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:12:48 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190469/ [7190470] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:33:37 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190470/ [7190471] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:18:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190471/ [7190472] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/mgr} time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190472/ [7190473] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:23:30 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190473/ [7190474] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:36:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190474/ [7190475] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/cache-pool-snaps} time: 00:24:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190475/ [7190476] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:23:56 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190476/ [7190477] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-balanced} time: 00:21:59 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190477/ [7190478] rados/singleton/{all/admin-socket mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190478/ [7190479] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:35:14 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190479/ [7190480] rados/objectstore/{backends/filestore-idempotent supported-random-distro$/{ubuntu_latest}} time: 02:43:43 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190480/ [7190481] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache-snaps-balanced} time: 00:23:32 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190481/ [7190482] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:01 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190482/ [7190483] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mix} time: 00:21:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190483/ [7190484] rados/cephadm/workunits/{0-distro/rhel_8.4_container_tools_3.0 agent/off mon_election/classic task/test_adoption} time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190484/ [7190485] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:23:58 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190485/ [7190486] rados/singleton/{all/backfill-toofull mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:40:39 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190486/ [7190487] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:19:09 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190487/ [7190489] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-snaps} time: 00:23:11 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190489/ [7190490] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:18:21 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190490/ [7190491] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/mapgap thrashosds-health workloads/rbd_cls} time: 00:24:23 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190491/ [7190492] rados/singleton/{all/deduptool mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:13:50 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190492/ [7190493] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:27:27 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190493/ [7190494] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:16:11 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190494/ [7190495] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:11:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190495/ [7190496] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:13:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190496/ [7190497] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:53 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190497/ [7190498] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/cache} time: 00:17:28 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190498/ [7190499] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/progress} time: 00:23:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190499/ [7190500] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:14 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190500/ [7190501] rados/singleton/{all/divergent_priors mon_election/classic msgr-failures/none msgr/async-v2only objectstore/filestore-xfs rados supported-random-distro$/{rhel_8}} time: 00:13:42 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190501/ [7190503] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:30:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190503/ [7190504] rados/standalone/{supported-random-distro$/{rhel_8} workloads/misc} time: 01:10:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190504/ [7190505] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/dedup-io-mixed} time: 00:15:15 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190505/ [7190506] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_mostlyread} time: 00:30:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190506/ [7190507] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:10:29 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190507/ [7190508] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read} time: 00:25:23 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190508/ [7190509] rados/objectstore/{backends/fusestore supported-random-distro$/{centos_8}} time: 00:11:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190509/ [7190510] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:13:49 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190510/ [7190511] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:32:20 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190511/ [7190512] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190512/ [7190513] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:27:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190513/ [7190514] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/dedup-io-snaps} time: 00:18:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190514/ [7190515] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:15:47 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190515/ [7190516] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:21:58 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190516/ [7190517] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:13:19 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190517/ [7190518] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:58 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190518/ [7190519] rados/multimon/{clusters/3 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:08:10 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190519/ [7190520] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/pool-snaps-few-objects} time: 00:29:04 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190520/ [7190521] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{filestore-xfs} tasks/dashboard} time: 01:10:26 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190521/ [7190523] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:21:04 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190523/ [7190524] rados/upgrade/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:27:02 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190524/ [7190525] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/none msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 01:46:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190525/ [7190526] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:36:24 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190526/ [7190527] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190527/ [7190528] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_api_tests} time: 00:32:45 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190528/ [7190529] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:06:35 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190529/ [7190530] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:09:00 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190530/ [7190531] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/rados_api_tests} time: 00:36:24 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190531/ [7190532] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:14:48 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190532/ [7190533] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:13:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190533/ [7190534] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:31:40 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190534/ [7190535] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8}} time: 01:13:18 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190535/ [7190536] rados/cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:17:03 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190536/ [7190537] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:12:12 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190537/ [7190538] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:38:37 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190538/ [7190539] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{centos_8}} time: 00:16:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190539/ [7190540] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/filestore-xfs rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:34:43 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190540/ [7190541] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-snappy rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-many-deletes} time: 00:24:05 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190541/ [7190542] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:15:59 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190542/ [7190543] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:17:34 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190543/ [7190544] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} time: 01:09:25 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190544/ [7190545] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:11:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190545/ [7190546] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:50 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190546/ [7190547] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/radosbench} time: 01:11:13 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190547/ [7190548] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190548/ [7190549] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/none msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8}} time: 00:50:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190549/ [7190550] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:09:26 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190550/ [7190551] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} time: 00:11:08 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190551/ [7190552] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_recovery} time: 00:13:05 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190552/ [7190553] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/repair_test} time: 00:19:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190553/ [7190554] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/redirect} time: 00:16:06 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190554/ [7190555] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_nfs} time: 00:30:15 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190555/ [7190556] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:15:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190556/ [7190557] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:14:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190557/ [7190558] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 01:02:23 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190558/ [7190559] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 00:55:31 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190559/ [7190560] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:05 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190560/ [7190561] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:27:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190561/ [7190562] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190562/ [7190563] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/mapgap thrashosds-health workloads/redirect_promote_tests} time: 00:23:05 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190563/ [7190564] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:15:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190564/ [7190565] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8}} time: 00:12:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190565/ [7190566] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{centos_8}} time: 00:27:38 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190566/ [7190567] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:12:33 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190567/ [7190568] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:09:52 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190568/ [7190569] rados/cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:46 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190569/ [7190570] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-small-objects} time: 00:24:07 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190570/ [7190571] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/redirect_set_object} time: 00:13:28 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190571/ [7190572] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:10:39 info: http://pulpito.ceph.com/yuriw-2023-03-01_19:28:10-rados-wip-yuri3-testing-2023-03-01-0812-quincy-distro-default-smithi/7190572/