2023-05-31T07:19:42.663 INFO:root:teuthology version: 0.0.1.dev111+gfa97ddd 2023-05-31T07:19:42.665 INFO:teuthology.results:subset: '111/120000' 2023-05-31T07:19:42.665 INFO:teuthology.results:seed: '451' 2023-05-31T12:50:51.579 INFO:root:teuthology version: 0.0.1.dev111+gfa97ddd 2023-05-31T12:50:51.614 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-05-31T15:40:08.783 INFO:teuthology.results:Tests finished! gathering results... 2023-05-31T15:40:33.693 INFO:scrape:Found 311 jobs 2023-05-31T15:40:34.012 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291014/teuthology.log 2023-05-31T15:40:36.768 INFO:scrape:Found 11 distinct failure reasons 2023-05-31T15:40:36.769 INFO:scrape:Failure: "1685524217.458616 mon.a (mon.0) 505 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log 2023-05-31T15:40:36.769 INFO:scrape:1 jobs: ['7290931'] 2023-05-31T15:40:36.770 INFO:scrape:suites: ['clusters/{fixed-2', 'mon_election/classic', 'msgr-failures/many', 'msgr/async', 'objectstore/bluestore-low-osd-mem-target', 'openstack}', 'rados', 'rados/basic/{ceph', 'supported-random-distro$/{ubuntu_latest}', 'tasks/rados_cls_all}'] 2023-05-31T15:40:36.770 INFO:scrape: 2023-05-31T15:40:36.770 INFO:scrape:Failure: No module named 'tasks' 2023-05-31T15:40:36.771 INFO:scrape:5 jobs: ['7291294', '7291275', '7291296', '7291292', '7291267'] 2023-05-31T15:40:36.771 INFO:scrape:suites intersection: [] 2023-05-31T15:40:36.771 INFO:scrape:suites union: ['ceph', 'clusters/{fixed-2', 'fast/fast', 'mon_election/classic', 'msgr-failures/fastclose', 'objectstore/bluestore-comp', 'objectstore/bluestore-comp-lz4', 'objectstore/bluestore-hybrid', 'openstack', 'openstack}', 'rados', 'rados/perf/{ceph', 'rados/singleton-nomsgr/{all/crushdiff', 'rados/standalone/{supported-random-distro$/{centos_8}', 'rados/thrash-erasure-code-isa/{arch/x86_64', 'rados/thrash-erasure-code/{ceph', 'recovery-overrides/{default}', 'recovery-overrides/{more-async-recovery}', 'scheduler/dmclock_default_shards', 'settings/optimized', 'supported-random-distro$/{ubuntu_20.04}', 'supported-random-distro$/{ubuntu_latest}', 'supported-random-distro$/{ubuntu_latest}}', 'thrashers/default', 'thrashers/minsize_recovery', 'thrashosds-health', 'ubuntu_latest', 'workloads/ec-rados-plugin=isa-k=2-m=1}', 'workloads/ec-small-objects-many-deletes}', 'workloads/fio_4K_rand_read}', 'workloads/mon}'] 2023-05-31T15:40:36.771 INFO:scrape: 2023-05-31T15:40:36.772 INFO:scrape:Dead: None 2023-05-31T15:40:36.772 INFO:scrape:2 jobs: ['7291014', '7291374'] 2023-05-31T15:40:36.772 INFO:scrape: 2023-05-31T15:40:36.773 INFO:scrape:Failure: Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f0bb306cc5aa92090c59e1564b031d6fd4f1a89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' 2023-05-31T15:40:36.773 INFO:scrape:2 jobs: ['7291055', '7291366'] 2023-05-31T15:40:36.773 INFO:scrape:suites intersection: ['debug/mgr', 'rados/dashboard/{0-single-container-host', 'tasks/e2e}'] 2023-05-31T15:40:36.774 INFO:scrape:suites union: ['debug/mgr', 'mon_election/classic', 'mon_election/connectivity', 'rados/dashboard/{0-single-container-host', 'random-objectstore$/{bluestore-comp-snappy}', 'random-objectstore$/{bluestore-hybrid}', 'tasks/e2e}'] 2023-05-31T15:40:36.774 INFO:scrape: 2023-05-31T15:40:36.774 INFO:scrape:Failure: Command failed on smithi154 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-TestClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'" 2023-05-31T15:40:36.775 INFO:scrape:1 jobs: ['7291147'] 2023-05-31T15:40:36.775 INFO:scrape:suites: ['0-size-min-size-overrides/2-size-2-min-size', '1-install/pacific', 'backoff/peering', 'ceph', 'clusters/{openstack', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'rados', 'rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools}', 'thrashers/mapgap', 'thrashosds-health', 'three-plus-one}', 'workloads/rbd_cls}'] 2023-05-31T15:40:36.776 INFO:scrape: 2023-05-31T15:40:36.776 INFO:scrape:Failure: Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi043 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f0bb306cc5aa92090c59e1564b031d6fd4f1a89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' 2023-05-31T15:40:36.777 INFO:scrape:2 jobs: ['7291320', '7291227'] 2023-05-31T15:40:36.777 INFO:scrape:suites intersection: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'rados', 'test_envlibrados_for_rocksdb}'] 2023-05-31T15:40:36.778 INFO:scrape:suites union: ['mon_election/connectivity', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-comp-zstd', 'objectstore/bluestore-low-osd-mem-target', 'rados', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest', 'rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest', 'supported-random-distro$/{centos_8}}', 'supported-random-distro$/{ubuntu_20.04}}', 'test_envlibrados_for_rocksdb}'] 2023-05-31T15:40:36.779 INFO:scrape: 2023-05-31T15:40:36.779 INFO:scrape:Failure: Command failed on smithi031 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" 2023-05-31T15:40:36.779 INFO:scrape:4 jobs: ['7290902', '7291213', '7291370', '7291057'] 2023-05-31T15:40:36.780 INFO:scrape:suites intersection: ['0-kubeadm', '0-nvme-loop', '1-rook', 'k8s/1.21', 'rados/rook/smoke/{0-distro/ubuntu_20.04'] 2023-05-31T15:40:36.780 INFO:scrape:suites union: ['0-kubeadm', '0-nvme-loop', '1-rook', '2-workload/none', '2-workload/radosbench', 'cluster/1-node', 'cluster/3-node', 'k8s/1.21', 'net/calico', 'net/flannel', 'net/host', 'rados/rook/smoke/{0-distro/ubuntu_20.04', 'rook/1.7.2}', 'rook/master}'] 2023-05-31T15:40:36.780 INFO:scrape: 2023-05-31T15:40:36.781 INFO:scrape:Failure: {'smithi186.front.sepia.ceph.com': {'_ansible_no_log': False, 'cache_update_time': 1685531440, 'cache_updated': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'python-dev', 'libfcgi0ldbl', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'only_upgrade': False, 'package': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'python-dev', 'libfcgi0ldbl', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'qemu-system-x86\' \'git-core\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.17-0ubuntu0.20.04.4_amd64.deb Connection failed [IP: 91.189.91.39 80]\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.17-0ubuntu0.20.04.4_amd64.deb Connection failed [IP: 91.189.91.39 80]\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.17-0ubuntu0.20.04.4_amd64.deb Connection failed [IP: 91.189.91.39 80]', 'E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?'], 'stdout': 'Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git qemu-system-data qemu-system-gui\n3 upgraded, 6 newly installed, 0 to remove and 184 not upgraded.\nNeed to get 18.9 MB of archives.\nAfter this operation, 63.1 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.11 [4605 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4 [3227 kB]\nErr:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4\n Connection failed [IP: 91.189.91.39 80]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.17-0ubuntu0.20.04.4 [1625 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.26 [54.5 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.26 [1057 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.26 [563 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.26 [40.5 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.26 [6726 kB]\nGet:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.26 [968 kB]\nFetched 15.6 MB in 35s (449 kB/s)\n', 'stdout_lines': ['Reading package lists...', 'Building dependency tree...', 'Reading state information...', 'The following additional packages will be installed:', ' librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-data', ' qemu-system-gui qemu-utils', 'Suggested packages:', ' git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk', ' gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap', 'The following NEW packages will be installed:', ' librados2 librbd1 qemu-block-extra qemu-system-common qemu-system-x86', ' qemu-utils', 'The following packages will be upgraded:', ' git qemu-system-data qemu-system-gui', '3 upgraded, 6 newly installed, 0 to remove and 184 not upgraded.', 'Need to get 18.9 MB of archives.', 'After this operation, 63.1 MB of additional disk space will be used.', 'Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.11 [4605 kB]', 'Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4 [3227 kB]', 'Err:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4', ' Connection failed [IP: 91.189.91.39 80]', 'Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.17-0ubuntu0.20.04.4 [1625 kB]', 'Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.26 [54.5 kB]', 'Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-common amd64 1:4.2-3ubuntu6.26 [1057 kB]', 'Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.26 [563 kB]', 'Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.26 [40.5 kB]', 'Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.26 [6726 kB]', 'Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.26 [968 kB]', 'Fetched 15.6 MB in 35s (449 kB/s)']}} 2023-05-31T15:40:36.781 INFO:scrape:1 jobs: ['7291191'] 2023-05-31T15:40:36.781 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'rados/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_orch_cli_mon}'] 2023-05-31T15:40:36.782 INFO:scrape: 2023-05-31T15:40:36.782 INFO:scrape:Failure: failed to install new kernel version within timeout 2023-05-31T15:40:36.782 INFO:scrape:1 jobs: ['7291025'] 2023-05-31T15:40:36.783 INFO:scrape:suites: ['1-pg-log-overrides/short_pg_log', '2-recovery-overrides/{more-active-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-3}', 'backoff/normal', 'ceph', 'clusters/{fixed-2', 'crc-failures/default', 'd-balancer/on', 'mon_election/connectivity', 'msgr-failures/fastclose', 'msgr/async-v1only', 'objectstore/bluestore-comp-zlib', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/3-size-2-min-size', 'supported-random-distro$/{ubuntu_20.04}', 'thrashers/morepggrow', 'thrashosds-health', 'workloads/small-objects-balanced}'] 2023-05-31T15:40:36.783 INFO:scrape: 2023-05-31T15:40:36.783 INFO:scrape:Failure: Command failed on smithi046 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' 2023-05-31T15:40:36.783 INFO:scrape:1 jobs: ['7291125'] 2023-05-31T15:40:36.784 INFO:scrape:suites: ['mon_election/connectivity', 'rados', 'rados/singleton-nomsgr/{all/multi-backfill-reject', 'supported-random-distro$/{ubuntu_latest}}'] 2023-05-31T15:40:36.784 INFO:scrape: 2023-05-31T15:40:36.784 INFO:scrape:Failure: Command failed on smithi031 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-3538-g8f0bb306-1jammy ceph-mds=18.0.0-3538-g8f0bb306-1jammy ceph-common=18.0.0-3538-g8f0bb306-1jammy ceph-fuse=18.0.0-3538-g8f0bb306-1jammy ceph-test=18.0.0-3538-g8f0bb306-1jammy radosgw=18.0.0-3538-g8f0bb306-1jammy python-ceph=18.0.0-3538-g8f0bb306-1jammy libcephfs1=18.0.0-3538-g8f0bb306-1jammy libcephfs-java=18.0.0-3538-g8f0bb306-1jammy libcephfs-jni=18.0.0-3538-g8f0bb306-1jammy librados2=18.0.0-3538-g8f0bb306-1jammy librbd1=18.0.0-3538-g8f0bb306-1jammy rbd-fuse=18.0.0-3538-g8f0bb306-1jammy' 2023-05-31T15:40:36.785 INFO:scrape:1 jobs: ['7291260'] 2023-05-31T15:40:36.785 INFO:scrape:suites: ['1-pg-log-overrides/normal_pg_log', '2-recovery-overrides/{more-async-recovery}', '3-scrub-overrides/{max-simultaneous-scrubs-2}', 'backoff/peering', 'ceph', 'clusters/{fixed-2', 'crc-failures/bad_map_crc_failure', 'd-balancer/crush-compat', 'mon_election/classic', 'msgr-failures/few', 'msgr/async-v2only', 'objectstore/bluestore-bitmap', 'openstack}', 'rados', 'rados/thrash/{0-size-min-size-overrides/2-size-2-min-size', 'supported-random-distro$/{ubuntu_latest}', 'thrashers/none', 'thrashosds-health', 'workloads/cache-snaps}'] 2023-05-31T15:40:36.785 INFO:scrape: 2023-05-31T15:40:36.798 INFO:teuthology.results:Sending results to yweinste@redhat.com: Test Run: yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/ failed: 18 dead: 3 running: 0 waiting: 0 queued: 0 passed: 290 Fail ================================================================= [7290902] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/calico rook/1.7.2} ----------------------------------------------------------------- time: 00:07:00 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290902/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290902/ sentry: https://sentry.ceph.com/organizations/ceph/?query=da87406e21b34453843f5980fe31bf2b Command failed on smithi031 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7290931] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/rados_cls_all} ----------------------------------------------------------------- time: 00:36:22 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290931/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290931/ "1685524217.458616 mon.a (mon.0) 505 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log [7291025] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/small-objects-balanced} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291025/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291025/ sentry: https://sentry.ceph.com/organizations/ceph/?query=da5f8cd9787a4c2685287b326570c229 failed to install new kernel version within timeout [7291055] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-hybrid} tasks/e2e} ----------------------------------------------------------------- time: 00:19:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291055/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291055/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6f81744bff74469abefc00d2c081ef32 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi062 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f0bb306cc5aa92090c59e1564b031d6fd4f1a89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7291057] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/flannel rook/master} ----------------------------------------------------------------- time: 00:08:46 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291057/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291057/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e886ab99e8f4c1c9d5e1eceecad5b49 Command failed on smithi016 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7291125] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291125/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291125/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2a343f3b323749df9ea935fa7ff9299d Command failed on smithi046 with status 1: 'sudo fuser -v /var/lib/dpkg/lock-frontend' [7291147] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/pacific backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/few rados thrashers/mapgap thrashosds-health workloads/rbd_cls} ----------------------------------------------------------------- time: 00:22:09 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291147/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291147/ sentry: https://sentry.ceph.com/organizations/ceph/?query=be57d9a12f7544c7acd0d6b0f043e398 Command failed on smithi154 with status 1: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph_test_cls_rbd --gtest_filter=-Te stClsRbd.get_features:TestClsRbd.parents:TestClsRbd.mirror'" [7291213] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/radosbench cluster/1-node k8s/1.21 net/host rook/1.7.2} ----------------------------------------------------------------- time: 00:07:54 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291213/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291213/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3c329d8ef45f4bb687dfb348cd504294 Command failed on smithi067 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" [7291227] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/centos_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} ----------------------------------------------------------------- time: 00:13:39 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291227/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291227/ sentry: https://sentry.ceph.com/organizations/ceph/?query=fffd52b97e254ee0a783d060400c413b Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi086 with status 2: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f0bb306cc5aa92090c59e1564b031d6fd4f1a89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7291260] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/cache-snaps} ----------------------------------------------------------------- time: 00:08:32 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291260/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291260/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e84df374ae4d40ce8039ac897c638543 Command failed on smithi031 with status 100: 'sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::=" --force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-3538-g8f0bb306-1jammy ceph-mds=18.0.0-3538-g8f0bb306-1jammy ceph-common=18.0.0-3538-g8f0bb306-1jammy ceph- fuse=18.0.0-3538-g8f0bb306-1jammy ceph-test=18.0.0-3538-g8f0bb306-1jammy radosgw=18.0.0-3538-g8f0bb306-1jammy python- ceph=18.0.0-3538-g8f0bb306-1jammy libcephfs1=18.0.0-3538-g8f0bb306-1jammy libcephfs-java=18.0.0-3538-g8f0bb306-1jammy libcephfs- jni=18.0.0-3538-g8f0bb306-1jammy librados2=18.0.0-3538-g8f0bb306-1jammy librbd1=18.0.0-3538-g8f0bb306-1jammy rbd-fuse=18.0.0-3538-g8f0bb306-1jammy' [7291267] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_read} ----------------------------------------------------------------- time: 00:12:49 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291267/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291267/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4e5ae028fb3940f1993f41e84c5c27be No module named 'tasks.ceph' [7291275] rados/standalone/{supported-random-distro$/{centos_8} workloads/mon} ----------------------------------------------------------------- time: 00:10:08 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291275/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291275/ sentry: https://sentry.ceph.com/organizations/ceph/?query=009f18e925514426b2c3b0a93bb02d0d No module named 'tasks.workunit' [7291292] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} ----------------------------------------------------------------- time: 00:09:46 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291292/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291292/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5370c3b9e28c481e961f12a34f6bc2d0 No module named 'tasks' [7291294] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/minsize_recovery thrashosds-health workloads/ec-small-objects-many-deletes} ----------------------------------------------------------------- time: 00:12:23 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291294/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291294/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c6119d51fa894661a8f29f29b3ada7fc No module named 'tasks' [7291296] rados/singleton-nomsgr/{all/crushdiff mon_election/classic rados supported-random-distro$/{ubuntu_latest}} ----------------------------------------------------------------- time: 00:09:39 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291296/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291296/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6c3074aea53c4f8780133a400a788900 No module named 'tasks' [7291320] rados/singleton/{all/test_envlibrados_for_rocksdb/{supported/rhel_latest test_envlibrados_for_rocksdb} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8}} ----------------------------------------------------------------- time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291320/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291320/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1c5430f492f14f4d9020d4d643fbf5e5 Command failed (workunit test rados/test_envlibrados_for_rocksdb.sh) on smithi043 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f0bb306cc5aa92090c59e1564b031d6fd4f1a89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/rados/test_envlibrados_for_rocksdb.sh' [7291366] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-comp-snappy} tasks/e2e} ----------------------------------------------------------------- time: 00:23:54 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291366/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291366/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f0da48eb852c4f7bae03c27bfb6ffec0 Command failed (workunit test cephadm/test_dashboard_e2e.sh) on smithi072 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=8f0bb306cc5aa92090c59e1564b031d6fd4f1a89 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clo ne.client.0/qa/workunits/cephadm/test_dashboard_e2e.sh' [7291370] rados/rook/smoke/{0-distro/ubuntu_20.04 0-kubeadm 0-nvme-loop 1-rook 2-workload/none cluster/3-node k8s/1.21 net/calico rook/master} ----------------------------------------------------------------- time: 00:09:02 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291370/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291370/ sentry: https://sentry.ceph.com/organizations/ceph/?query=bf0d910b90c74aef8a8474f2c7dc5091 Command failed on smithi002 with status 100: "sudo apt update && sudo apt install -y apt-transport-https ca-certificates curl && sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg && echo 'deb [signed- by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main' | sudo tee /etc/apt/sources.list.d/kubernetes.list && sudo apt update && sudo apt install -y kubelet kubeadm kubectl bridge-utils" Dead ================================================================= [7291014] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/rados_stress_watch} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291014/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291014/ Error reimaging machines: Failed to power on smithi019 [7291191] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} ----------------------------------------------------------------- time: 00:08:52 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291191/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291191/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1b2577b8e233402594b15d34519546a1 {'smithi186.front.sepia.ceph.com': {'_ansible_no_log': False, 'cache_update_time': 1685531440, 'cache_updated': False, 'changed': False, 'invocation': {'module_args': {'allow_unauthenticated': False, 'autoclean': False, 'autoremove': False, 'cache_valid_time': 0, 'deb': None, 'default_release': None, 'dpkg_options': 'force-confdef,force-confold', 'force': True, 'force_apt_get': False, 'install_recommends': None, 'name': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'python-dev', 'libfcgi0ldbl', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid-dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default-jdk', 'junit4', 'tgt', 'open- iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'only_upgrade': False, 'package': ['mpich', 'qemu-system-x86', 'lttng-tools', 'libtool-bin', 'docker.io', 'python3-nose', 'python3-venv', 'python3-virtualenv', 'python3-configobj', 'python3-gevent', 'python3-numpy', 'python3-matplotlib', 'python3-setuptools', 'python-dev', 'libfcgi0ldbl', 'libev-dev', 'perl', 'libwww-perl', 'lsb-release', 'build-essential', 'sysstat', 'gdb', 'libedit2', 'cryptsetup-bin', 'xfsprogs', 'gdisk', 'parted', 'libuuid1', 'libatomic-ops-dev', 'git-core', 'attr', 'dbench', 'bonnie++', 'valgrind', 'ant', 'libtool', 'automake', 'gettext', 'uuid- dev', 'libacl1-dev', 'bc', 'xfsdump', 'xfslibs-dev', 'libattr1-dev', 'quota', 'libcap2-bin', 'libncurses5-dev', 'lvm2', 'vim', 'pdsh', 'blktrace', 'genisoimage', 'libjson-xs-perl', 'xml-twig-tools', 'default- jdk', 'junit4', 'tgt', 'open-iscsi', 'cifs-utils', 'ipcalc', 'nfs-common', 'nfs-kernel-server', 'software-properties-common', 'libgnutls30'], 'policy_rc_d': None, 'purge': False, 'state': 'present', 'update_cache': None, 'update_cache_retries': 5, 'update_cache_retry_max_delay': 12, 'upgrade': None}}, 'msg': '\'/usr/bin/apt-get -y -o "Dpkg::Options::=--force-confdef" -o "Dpkg::Options::=--force-confold" --force-yes install \'qemu-system-x86\' \'git-core\'\' failed: W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/c eph/librados2_15.2.17-0ubuntu0.20.04.4_amd64.deb Connection failed [IP: 91.189.91.39 80]\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'rc': 100, 'stderr': 'W: --force-yes is deprecated, use one of the options starting with --allow instead.\nE: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2 _15.2.17-0ubuntu0.20.04.4_amd64.deb Connection failed [IP: 91.189.91.39 80]\nE: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?\n', 'stderr_lines': ['W: --force-yes is deprecated, use one of the options starting with --allow instead.', 'E: Failed to fetch http:// archive.ubuntu.com/ubuntu/pool/main/c/ceph/librados2_15.2.17-0ubuntu0.20.04 .4_amd64.deb Connection failed [IP: 91.189.91.39 80]', 'E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?'], 'stdout': 'Reading package lists...\nBuilding dependency tree...\nReading state information...\nThe following additional packages will be installed:\n librados2 librbd1 qemu-block-extra qemu-system-common qemu- system-data\n qemu-system-gui qemu-utils\nSuggested packages:\n git- daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk\n gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap\nThe following NEW packages will be installed:\n librados2 librbd1 qemu-block-extra qemu- system-common qemu-system-x86\n qemu-utils\nThe following packages will be upgraded:\n git qemu-system-data qemu-system-gui\n3 upgraded, 6 newly installed, 0 to remove and 184 not upgraded.\nNeed to get 18.9 MB of archives.\nAfter this operation, 63.1 MB of additional disk space will be used.\nGet:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.11 [4605 kB]\nGet:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4 [3227 kB]\nErr:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4\n Connection failed [IP: 91.189.91.39 80]\nGet:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librbd1 amd64 15.2.17-0ubuntu0.20.04.4 [1625 kB]\nGet:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-block-extra amd64 1:4.2-3ubuntu6.26 [54.5 kB]\nGet:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu- system-common amd64 1:4.2-3ubuntu6.26 [1057 kB]\nGet:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.26 [563 kB]\nGet:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.26 [40.5 kB]\nGet:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu- system-x86 amd64 1:4.2-3ubuntu6.26 [6726 kB]\nGet:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.26 [968 kB]\nFetched 15.6 MB in 35s (449 kB/s)\n', 'stdout_lines': ['Reading package lists...', 'Building dependency tree...', 'Reading state information...', 'The following additional packages will be installed:', ' librados2 librbd1 qemu-block-extra qemu-system-common qemu- system-data', ' qemu-system-gui qemu-utils', 'Suggested packages:', ' git-daemon-run | git-daemon-sysvinit git-doc git-el git-email git-gui gitk', ' gitweb git-cvs git-mediawiki git-svn samba vde2 debootstrap', 'The following NEW packages will be installed:', ' librados2 librbd1 qemu- block-extra qemu-system-common qemu-system-x86', ' qemu-utils', 'The following packages will be upgraded:', ' git qemu-system-data qemu-system- gui', '3 upgraded, 6 newly installed, 0 to remove and 184 not upgraded.', 'Need to get 18.9 MB of archives.', 'After this operation, 63.1 MB of additional disk space will be used.', 'Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 git amd64 1:2.25.1-1ubuntu3.11 [4605 kB]', 'Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4 [3227 kB]', 'Err:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 librados2 amd64 15.2.17-0ubuntu0.20.04.4', ' Connection failed [IP: 91.189.91.39 80]', 'Get:3 http://archive.ubuntu.com/ubuntu focal- updates/main amd64 librbd1 amd64 15.2.17-0ubuntu0.20.04.4 [1625 kB]', 'Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu- block-extra amd64 1:4.2-3ubuntu6.26 [54.5 kB]', 'Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system- common amd64 1:4.2-3ubuntu6.26 [1057 kB]', 'Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-data all 1:4.2-3ubuntu6.26 [563 kB]', 'Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-gui amd64 1:4.2-3ubuntu6.26 [40.5 kB]', 'Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-system-x86 amd64 1:4.2-3ubuntu6.26 [6726 kB]', 'Get:9 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 qemu-utils amd64 1:4.2-3ubuntu6.26 [968 kB]', 'Fetched 15.6 MB in 35s (449 kB/s)']}} [7291374] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291374/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291374/ Error reimaging machines: reached maximum tries (120) after waiting for 1800 seconds Pass ================================================================= [7290817] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/force-sync-many workloads/rados_mon_osdmap_prune} time: 00:48:30 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290817/ [7290818] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/cache-snaps-balanced} time: 00:28:33 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290818/ [7290819] rados/singleton/{all/divergent_priors2 mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:13:19 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290819/ [7290820] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:14:33 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290820/ [7290821] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v2only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/snaps-few-objects} time: 00:33:58 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290821/ [7290822] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:08:29 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290822/ [7290823] rados/multimon/{clusters/6 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:22:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290823/ [7290824] rados/singleton-nomsgr/{all/msgr mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:18:45 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290824/ [7290825] rados/singleton/{all/dump-stuck mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290825/ [7290826] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:16:35 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290826/ [7290827] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-snaps} time: 00:23:05 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290827/ [7290828] rados/objectstore/{backends/objectstore-memstore supported-random-distro$/{ubuntu_latest}} time: 00:12:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290828/ [7290829] rados/singleton-nomsgr/{all/multi-backfill-reject mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:24:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290829/ [7290830] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/fastread thrashosds-health workloads/ec-small-objects} time: 00:26:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290830/ [7290831] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:18:01 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290831/ [7290832] rados/singleton/{all/ec-inconsistent-hinfo mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:22:54 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290832/ [7290833] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:57 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290833/ [7290834] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/cache} time: 00:17:35 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290834/ [7290835] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{more-async-recovery} supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:18:37 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290835/ [7290836] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} tasks/libcephsqlite} time: 00:20:48 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290836/ [7290837] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:17:47 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290837/ [7290838] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:20:28 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290838/ [7290839] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados tasks/rados_cls_all validater/valgrind} time: 02:08:38 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290839/ [7290840] rados/perf/{ceph mon_election/classic objectstore/bluestore-comp openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:15:25 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290840/ [7290841] rados/singleton/{all/ec-lost-unfound mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest}} time: 00:36:59 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290841/ [7290842] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:22:24 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290842/ [7290843] rados/standalone/{supported-random-distro$/{rhel_8} workloads/scrub} time: 02:46:41 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290843/ [7290844] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-zlib} supported-random-distro$/{ubuntu_latest} tasks/workunits} time: 00:14:24 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290844/ [7290845] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:19:42 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290845/ [7290846] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/dedup-io-mixed} time: 00:16:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290846/ [7290847] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:27:55 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290847/ [7290848] rados/singleton-nomsgr/{all/pool-access mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:11:45 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290848/ [7290850] rados/singleton/{all/erasure-code-nonregression mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:14 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290850/ [7290852] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_workunits} time: 00:36:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290852/ [7290854] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_nfs} time: 00:31:27 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290854/ [7290856] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/dedup-io-snaps} time: 00:15:59 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290856/ [7290858] rados/multimon/{clusters/9 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-stupid rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:15:33 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290858/ [7290860] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:30:23 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290860/ [7290862] rados/singleton/{all/lost-unfound-delete mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:32:49 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290862/ [7290864] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:27:58 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290864/ [7290866] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_write} time: 00:16:07 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290866/ [7290868] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} tasks/rados_api_tests} time: 00:35:02 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290868/ [7290870] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/pool-snaps-few-objects} time: 00:36:54 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290870/ [7290872] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli} time: 00:20:57 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290872/ [7290874] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/nautilus backoff/peering ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/few rados thrashers/pggrow thrashosds-health workloads/test_rbd_api} time: 00:25:20 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290874/ [7290876] rados/singleton/{all/lost-unfound mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:34:55 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290876/ [7290878] rados/singleton-nomsgr/{all/version-number-sanity mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:25 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290878/ [7290880] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/minsize_recovery thrashosds-health workloads/ec-rados-plugin=clay-k=4-m=2} time: 00:34:21 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290880/ [7290882] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/none msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:38:59 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290882/ [7290884] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:32:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290884/ [7290886] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:44 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290886/ [7290888] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:34:44 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290888/ [7290890] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} thrashers/mapgap thrashosds-health workloads/rados_api_tests} time: 00:29:08 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290890/ [7290892] rados/singleton/{all/max-pg-per-osd.from-mon mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:10:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290892/ [7290894] rados/dashboard/{0-single-container-host debug/mgr mon_election/classic random-objectstore$/{bluestore-hybrid} tasks/dashboard} time: 01:12:47 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290894/ [7290896] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{ubuntu_latest} tasks/crash} time: 00:17:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290896/ [7290898] rados/objectstore/{backends/ceph_objectstore_tool supported-random-distro$/{ubuntu_latest}} time: 00:43:41 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290898/ [7290900] rados/rest/{mgr-restful supported-random-distro$/{ubuntu_latest}} time: 00:17:20 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290900/ [7290904] rados/singleton-nomsgr/{all/admin_socket_output mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:21:10 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290904/ [7290907] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/c2c} time: 00:17:39 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290907/ [7290909] rados/upgrade/parallel/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:26:58 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290909/ [7290911] rados/valgrind-leaks/{1-start 2-inject-leak/mon centos_latest} time: 00:27:00 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290911/ [7290913] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados tasks/mon_recovery validater/lockdep} time: 00:15:10 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290913/ [7290915] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli_mon} time: 00:30:38 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290915/ [7290917] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/few objectstore/bluestore-stupid rados recovery-overrides/{more-active-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290917/ [7290919] rados/singleton/{all/max-pg-per-osd.from-primary mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:17:01 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290919/ [7290921] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/radosbench-high-concurrency} time: 00:40:50 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290921/ [7290923] rados/singleton-nomsgr/{all/balancer mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290923/ [7290924] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:31 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290924/ [7290927] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4K_rand_read} time: 00:12:20 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290927/ [7290929] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/snaps-few-objects} time: 00:39:29 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290929/ [7290933] rados/singleton/{all/max-pg-per-osd.from-replica mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:16:16 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290933/ [7290934] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:16:05 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290934/ [7290936] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} tasks/mon_clock_with_skews} time: 00:16:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290936/ [7290938] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/none thrashosds-health workloads/radosbench} time: 00:37:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290938/ [7290940] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:24:06 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290940/ [7290942] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:13:05 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290942/ [7290944] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:24 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290944/ [7290946] rados/singleton/{all/mon-auth-caps mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04}} time: 00:15:27 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290946/ [7290948] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/pggrow thrashosds-health workloads/redirect} time: 00:19:58 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290948/ [7290950] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:19:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290950/ [7290952] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=2-m=1} time: 00:30:14 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290952/ [7290954] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_rgw_multisite} time: 00:18:50 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290954/ [7290956] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:47 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290956/ [7290958] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-zstd} supported-random-distro$/{centos_8} tasks/failover} time: 00:16:25 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290958/ [7290960] rados/singleton/{all/mon-config-key-caps mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04}} time: 00:15:37 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290960/ [7290962] rados/standalone/{supported-random-distro$/{rhel_8} workloads/crush} time: 00:30:23 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290962/ [7290964] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zstd rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:21:51 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290964/ [7290966] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4K_seq_read} time: 00:14:46 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290966/ [7290968] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:14:56 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290968/ [7290970] rados/singleton-nomsgr/{all/ceph-post-file mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:12:41 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290970/ [7290972] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/redirect_promote_tests} time: 00:20:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290972/ [7290974] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/osd-delay rados thrashers/careful thrashosds-health workloads/cache-snaps} time: 00:32:34 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290974/ [7290976] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados tasks/rados_api_tests validater/valgrind} time: 00:49:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290976/ [7290978] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/rados_python} time: 00:18:32 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290978/ [7290980] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-bitmap rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:28:05 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290980/ [7290982] rados/singleton/{all/mon-config-keys mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:23:29 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290982/ [7290984] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_set_mon_crush_locations} time: 00:33:20 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290984/ [7290986] rados/singleton-nomsgr/{all/crushdiff mon_election/connectivity rados supported-random-distro$/{rhel_8}} time: 00:24:13 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290986/ [7290988] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/redirect_set_object} time: 00:27:07 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290988/ [7290990] rados/objectstore/{backends/fusestore supported-random-distro$/{rhel_8}} time: 00:20:07 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290990/ [7290992] rados/singleton/{all/mon-config mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:13:03 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290992/ [7290994] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04} thrashers/sync-many workloads/pool-create-delete} time: 00:18:35 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290994/ [7290996] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:19:40 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290996/ [7290998] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async-v2only no_pools objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8} tasks/mon_recovery} time: 00:36:26 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7290998/ [7291000] rados/singleton-nomsgr/{all/export-after-evict mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:12:53 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291000/ [7291002] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_adoption} time: 00:18:47 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291002/ [7291004] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/set-chunks-read} time: 00:20:34 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291004/ [7291006] rados/perf/{ceph mon_election/classic objectstore/bluestore-bitmap openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_rand_read} time: 00:13:28 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291006/ [7291008] rados/singleton/{all/osd-backfill mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{centos_8}} time: 00:18:30 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291008/ [7291010] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{rhel_8} tasks/insights} time: 00:23:26 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291010/ [7291012] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:20:42 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291012/ [7291016] rados/singleton-nomsgr/{all/full-tiering mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:13:11 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291016/ [7291018] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/fastclose objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=3-m=1} time: 00:31:05 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291018/ [7291020] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-small-objects-overwrites} time: 00:26:02 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291020/ [7291022] rados/singleton/{all/osd-recovery-incomplete mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8}} time: 00:26:30 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291022/ [7291027] rados/standalone/{supported-random-distro$/{ubuntu_latest} workloads/erasure-code} time: 01:29:20 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291027/ [7291028] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{rhel_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:52 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291028/ [7291030] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm} time: 00:20:13 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291030/ [7291033] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-hybrid rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:30:51 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291033/ [7291035] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:38:47 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291035/ [7291037] rados/singleton/{all/osd-recovery mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8}} time: 00:16:50 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291037/ [7291039] rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:17:17 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291039/ [7291041] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-lz4 rados tasks/rados_cls_all validater/lockdep} time: 00:21:07 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291041/ [7291043] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/none thrashosds-health workloads/small-objects-localized} time: 00:26:45 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291043/ [7291045] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:19:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291045/ [7291046] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:33 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291046/ [7291048] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-comp openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/radosbench_4M_seq_read} time: 00:13:17 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291048/ [7291051] rados/singleton/{all/peer mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:56 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291051/ [7291053] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:17:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291053/ [7291059] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:16:38 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291059/ [7291061] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/octopus backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/fastclose rados thrashers/default thrashosds-health workloads/radosbench} time: 01:31:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291061/ [7291063] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} thrashers/pggrow thrashosds-health workloads/small-objects} time: 00:23:54 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291063/ [7291065] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v2only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/sync workloads/rados_5925} time: 00:15:39 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291065/ [7291067] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:08:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291067/ [7291069] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04} tasks/rados_striper} time: 00:14:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291069/ [7291070] rados/singleton/{all/pg-autoscaler-progress-off mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest}} time: 00:18:24 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291070/ [7291072] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async no_pools objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} tasks/mon_clock_no_skews} time: 00:15:14 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291072/ [7291074] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:15:22 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291074/ [7291077] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/module_selftest} time: 00:34:38 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291077/ [7291079] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/snaps-few-objects-balanced} time: 00:33:24 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291079/ [7291080] rados/objectstore/{backends/keyvaluedb supported-random-distro$/{rhel_8}} time: 00:21:15 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291080/ [7291083] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools} 1-start 2-services/rgw 3-final} time: 00:22:02 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291083/ [7291085] rados/singleton/{all/pg-autoscaler mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:25:25 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291085/ [7291087] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mgr} time: 00:22:26 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291087/ [7291089] rados/singleton-nomsgr/{all/librados_hello_world mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:18:16 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291089/ [7291091] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/careful thrashosds-health workloads/ec-radosbench} time: 01:17:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291091/ [7291093] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:22:05 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291093/ [7291095] rados/perf/{ceph mon_election/classic objectstore/bluestore-low-osd-mem-target openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/radosbench_4M_write} time: 00:13:10 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291095/ [7291097] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:06 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291097/ [7291099] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/snaps-few-objects-localized} time: 00:32:50 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291099/ [7291101] rados/singleton/{all/pg-removal-interruption mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:17:37 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291101/ [7291103] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:17:28 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291103/ [7291105] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291105/ [7291107] rados/singleton-nomsgr/{all/msgr mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:23:49 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291107/ [7291109] rados/valgrind-leaks/{1-start 2-inject-leak/none centos_latest} time: 00:22:07 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291109/ [7291111] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-snappy rados tasks/mon_recovery validater/valgrind} time: 00:31:22 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291111/ [7291113] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-dispatch-delay rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-pool-snaps-few-objects-overwrites} time: 00:36:33 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291113/ [7291115] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291115/ [7291117] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/snaps-few-objects} time: 00:34:47 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291117/ [7291119] rados/singleton/{all/radostool mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{centos_8}} time: 00:17:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291119/ [7291121] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest} tasks/rados_workunit_loadgen_big} time: 00:35:41 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291121/ [7291123] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:27:13 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291123/ [7291127] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs} time: 00:32:51 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291127/ [7291129] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_latest} thrashers/force-sync-many workloads/rados_api_tests} time: 00:35:40 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291129/ [7291131] rados/singleton/{all/random-eio mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:32:48 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291131/ [7291133] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest} thrashers/morepggrow thrashosds-health workloads/write_fadvise_dontneed} time: 00:21:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291133/ [7291135] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-bitmap} supported-random-distro$/{centos_8} tasks/per_module_finisher_stats} time: 00:11:52 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291135/ [7291137] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-stupid openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/radosbench_omap_write} time: 00:26:03 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291137/ [7291139] rados/multimon/{clusters/6 mon_election/classic msgr-failures/few msgr/async-v1only no_pools objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} tasks/mon_clock_with_skews} time: 00:13:06 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291139/ [7291141] rados/singleton-nomsgr/{all/osd_stale_reads mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:21:27 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291141/ [7291143] rados/cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:20:16 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291143/ [7291145] rados/singleton/{all/rebuild-mondb mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{rhel_8}} time: 00:19:01 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291145/ [7291149] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/misc} time: 01:09:21 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291149/ [7291151] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/admin_socket_objecter_requests} time: 00:22:30 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291151/ [7291153] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:39 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291153/ [7291155] rados/singleton-nomsgr/{all/pool-access mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:11:46 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291155/ [7291157] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/osd-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-small-objects-balanced} time: 00:23:25 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291157/ [7291159] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-comp-zlib rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:31:59 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291159/ [7291161] rados/singleton/{all/recovery-preemption mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_20.04}} time: 00:25:16 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291161/ [7291163] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mix} time: 00:32:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291163/ [7291165] rados/objectstore/{backends/objectcacher-stress supported-random-distro$/{ubuntu_20.04}} time: 00:34:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291165/ [7291167] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:20:10 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291167/ [7291169] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache-agent-big} time: 00:25:40 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291169/ [7291171] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-stupid rados recovery-overrides/{default} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:37:14 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291171/ [7291173] rados/singleton-nomsgr/{all/recovery-unfound-found mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:29:15 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291173/ [7291175] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-comp-zlib rados tasks/rados_api_tests validater/lockdep} time: 00:30:48 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291175/ [7291177] rados/singleton/{all/resolve_stuck_peering mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8}} time: 00:11:46 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291177/ [7291179] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/fastclose objectstore/bluestore-comp-zlib rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:19:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291179/ [7291181] rados/perf/{ceph mon_election/classic objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/sample_fio} time: 00:14:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291181/ [7291183] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:19:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291183/ [7291185] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-3} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/cache-agent-small} time: 00:18:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291185/ [7291186] rados/singleton-bluestore/{all/cephtool mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:34:48 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291186/ [7291189] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-low-osd-mem-target} supported-random-distro$/{centos_8} tasks/progress} time: 00:25:24 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291189/ [7291193] rados/singleton/{all/test-crash mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8}} time: 00:19:29 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291193/ [7291195] rados/singleton-nomsgr/{all/version-number-sanity mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:16:54 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291195/ [7291197] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/fastclose rados recovery-overrides/{more-active-recovery} supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/ec-small-objects-fast-read-overwrites} time: 00:31:26 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291197/ [7291199] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} thrashers/many workloads/rados_mon_osdmap_prune} time: 01:01:55 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291199/ [7291201] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v2only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8} thrashers/default thrashosds-health workloads/cache-pool-snaps-readproxy} time: 00:32:17 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291201/ [7291203] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:55 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291203/ [7291205] rados/singleton/{all/test-noautoscale-flag mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:15:40 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291205/ [7291207] rados/multimon/{clusters/9 mon_election/connectivity msgr-failures/many msgr/async-v2only no_pools objectstore/bluestore-comp-zstd rados supported-random-distro$/{centos_8} tasks/mon_recovery} time: 00:18:22 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291207/ [7291209] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{rhel_8} tasks/rados_workunit_loadgen_mostlyread} time: 00:34:17 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291209/ [7291211] rados/dashboard/{0-single-container-host debug/mgr mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} tasks/dashboard} time: 01:12:10 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291211/ [7291215] rados/singleton-nomsgr/{all/admin_socket_output mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:22:07 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291215/ [7291217] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/mon-stretch} time: 00:21:19 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291217/ [7291219] rados/upgrade/parallel/{0-random-distro$/{centos_8.stream_container_tools_crun} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:26:23 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291219/ [7291221] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 00:22:21 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291221/ [7291223] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/cache-pool-snaps} time: 00:31:31 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291223/ [7291225] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-bitmap openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/sample_radosbench} time: 00:11:42 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291225/ [7291229] rados/singleton-nomsgr/{all/balancer mon_election/classic rados supported-random-distro$/{ubuntu_latest}} time: 00:13:13 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291229/ [7291231] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-small-objects-fast-read} time: 00:23:01 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291231/ [7291233] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-comp-zstd rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:30:09 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291233/ [7291235] rados/cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_3.0} 1-start 2-services/basic 3-final} time: 00:15:45 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291235/ [7291237] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/3-size-2-min-size 1-install/quincy backoff/peering_and_degraded ceph clusters/{openstack three-plus-one} d-balancer/on mon_election/connectivity msgr-failures/osd-delay rados thrashers/morepggrow thrashosds-health workloads/snaps-few-objects} time: 00:37:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291237/ [7291239] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:18:11 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291239/ [7291241] rados/singleton/{all/thrash-backfill-full mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04}} time: 00:56:41 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291241/ [7291242] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/morepggrow thrashosds-health workloads/cache-snaps-balanced} time: 00:23:46 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291242/ [7291245] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:14:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291245/ [7291247] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-zstd rados recovery-overrides/{default} supported-random-distro$/{ubuntu_latest} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:18:27 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291247/ [7291248] rados/singleton-nomsgr/{all/cache-fs-trunc mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:16:07 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291248/ [7291250] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-comp-zstd rados tasks/rados_cls_all validater/valgrind} time: 02:08:24 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291250/ [7291252] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/classic random-objectstore$/{bluestore-comp-snappy} supported-random-distro$/{ubuntu_latest} tasks/prometheus} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291252/ [7291254] rados/singleton/{all/thrash-eio mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04}} time: 00:40:13 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291254/ [7291257] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:18:39 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291257/ [7291259] rados/objectstore/{backends/objectstore-bluestore-a supported-random-distro$/{rhel_8}} time: 02:19:34 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291259/ [7291263] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} tasks/readwrite} time: 00:14:26 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291263/ [7291265] rados/singleton-nomsgr/{all/ceph-kvstore-tool mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:22:13 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291265/ [7291269] rados/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:28:44 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291269/ [7291271] rados/monthrash/{ceph clusters/9-mons mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} thrashers/one workloads/rados_mon_workunits} time: 00:30:57 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291271/ [7291273] rados/singleton/{all/thrash-rados/{thrash-rados thrashosds-health} mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_latest}} time: 00:23:34 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291273/ [7291277] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-partial-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async objectstore/bluestore-comp-lz4 rados supported-random-distro$/{centos_8} thrashers/pggrow thrashosds-health workloads/cache} time: 00:19:44 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291277/ [7291279] rados/multimon/{clusters/21 mon_election/classic msgr-failures/few msgr/async no_pools objectstore/bluestore-hybrid rados supported-random-distro$/{ubuntu_20.04} tasks/mon_clock_no_skews} time: 00:16:42 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291279/ [7291281] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} time: 00:15:37 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291281/ [7291283] rados/singleton-nomsgr/{all/ceph-post-file mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:10:41 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291283/ [7291284] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/fast mon_election/classic msgr-failures/few rados recovery-overrides/{more-active-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-small-objects-overwrites} time: 00:23:33 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291284/ [7291286] rados/singleton/{all/thrash_cache_writeback_proxy_none mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{ubuntu_latest}} time: 00:25:47 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291286/ [7291288] rados/cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:46 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291288/ [7291290] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-partial-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v1only objectstore/bluestore-comp-snappy rados supported-random-distro$/{centos_8} thrashers/careful thrashosds-health workloads/dedup-io-mixed} time: 00:16:15 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291290/ [7291298] rados/singleton/{all/watch-notify-same-primary mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-zlib rados supported-random-distro$/{ubuntu_latest}} time: 00:11:35 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291298/ [7291300] rados/cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:11 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291300/ [7291302] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/repair_test} time: 00:21:44 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291302/ [7291304] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/connectivity msgr-failures/fastclose objectstore/bluestore-comp-lz4 rados recovery-overrides/{more-async-recovery} supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/ec-rados-plugin=jerasure-k=4-m=2} time: 00:29:20 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291304/ [7291306] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-low-osd-mem-target openstack scheduler/wpq_default_shards settings/optimized ubuntu_latest workloads/fio_4K_rand_rw} time: 00:13:01 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291306/ [7291308] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/enable mon_election/connectivity random-objectstore$/{bluestore-comp-lz4} supported-random-distro$/{centos_8} tasks/workunits} time: 00:17:03 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291308/ [7291310] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async-v2only objectstore/bluestore-comp-zlib rados supported-random-distro$/{centos_8} thrashers/default thrashosds-health workloads/dedup-io-snaps} time: 00:16:58 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291310/ [7291312] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/classic msgr-failures/osd-delay objectstore/bluestore-hybrid rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:17:29 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291312/ [7291314] rados/singleton-nomsgr/{all/export-after-evict mon_election/connectivity rados supported-random-distro$/{ubuntu_latest}} time: 00:14:30 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291314/ [7291316] rados/valgrind-leaks/{1-start 2-inject-leak/osd centos_latest} time: 00:27:42 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291316/ [7291318] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} time: 00:20:32 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291318/ [7291322] rados/cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} time: 00:21:02 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291322/ [7291324] rados/thrash-old-clients/{0-distro$/{centos_8.stream_container_tools} 0-size-min-size-overrides/2-size-2-min-size 1-install/nautilus-v1only backoff/normal ceph clusters/{openstack three-plus-one} d-balancer/crush-compat mon_election/classic msgr-failures/fastclose rados thrashers/none thrashosds-health workloads/test_rbd_api} time: 00:23:10 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291324/ [7291326] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-comp-zstd rados supported-random-distro$/{ubuntu_20.04} thrashers/mapgap thrashosds-health workloads/pool-snaps-few-objects} time: 00:25:16 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291326/ [7291328] rados/cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:28:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291328/ [7291330] rados/singleton/{all/admin-socket mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-hybrid rados supported-random-distro$/{rhel_8}} time: 00:14:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291330/ [7291332] rados/singleton-nomsgr/{all/full-tiering mon_election/classic rados supported-random-distro$/{centos_8}} time: 00:14:14 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291332/ [7291334] rados/standalone/{supported-random-distro$/{rhel_8} workloads/osd-backfill} time: 02:40:48 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291334/ [7291336] rados/singleton-bluestore/{all/cephtool mon_election/classic msgr-failures/many msgr/async objectstore/bluestore-comp-snappy rados supported-random-distro$/{rhel_8}} time: 00:42:18 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291336/ [7291338] rados/monthrash/{ceph clusters/3-mons mon_election/classic msgr-failures/mon-delay msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8} thrashers/sync-many workloads/pool-create-delete} time: 00:23:49 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291338/ [7291340] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:10 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291340/ [7291342] rados/objectstore/{backends/objectstore-bluestore-b supported-random-distro$/{ubuntu_latest}} time: 00:10:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291342/ [7291344] rados/multimon/{clusters/3 mon_election/connectivity msgr-failures/many msgr/async-v1only no_pools objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{ubuntu_latest} tasks/mon_clock_with_skews} time: 00:12:40 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291344/ [7291346] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/osd-delay msgr/async-v1only objectstore/bluestore-hybrid rados supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/rados_api_tests} time: 00:36:31 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291346/ [7291348] rados/singleton/{all/backfill-toofull mon_election/connectivity msgr-failures/none msgr/async-v1only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8}} time: 00:27:09 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291348/ [7291350] rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{ubuntu_20.04}} time: 00:14:00 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291350/ [7291352] rados/perf/{ceph mon_election/classic objectstore/bluestore-stupid openstack scheduler/dmclock_1Shard_16Threads settings/optimized ubuntu_latest workloads/fio_4M_rand_read} time: 00:15:08 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291352/ [7291354] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{rhel_8} tasks/scrub_test} time: 00:15:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291354/ [7291356] rados/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:07:43 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291356/ [7291358] rados/singleton/{all/deduptool mon_election/classic msgr-failures/few msgr/async-v2only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest}} time: 00:14:59 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291358/ [7291360] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{default} 3-scrub-overrides/{max-simultaneous-scrubs-2} backoff/peering ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/osd-dispatch-delay msgr/async-v2only objectstore/bluestore-low-osd-mem-target rados supported-random-distro$/{rhel_8} thrashers/none thrashosds-health workloads/radosbench-high-concurrency} time: 00:33:19 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291360/ [7291362] rados/thrash-erasure-code-isa/{arch/x86_64 ceph clusters/{fixed-2 openstack} mon_election/connectivity msgr-failures/few objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-async-partial-recovery} supported-random-distro$/{ubuntu_latest} thrashers/mapgap thrashosds-health workloads/ec-rados-plugin=isa-k=2-m=1} time: 00:29:08 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291362/ [7291364] rados/thrash-erasure-code/{ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{centos_8} thrashers/morepggrow thrashosds-health workloads/ec-small-objects} time: 00:24:09 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291364/ [7291368] rados/mgr/{clusters/{2-node-mgr} debug/mgr mgr_ttl_cache/disable mon_election/connectivity random-objectstore$/{bluestore-stupid} supported-random-distro$/{rhel_8} tasks/crash} time: 00:16:40 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291368/ [7291372] rados/singleton-nomsgr/{all/large-omap-object-warnings mon_election/classic rados supported-random-distro$/{ubuntu_20.04}} time: 00:11:23 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291372/ [7291376] rados/thrash-erasure-code-overwrites/{bluestore-bitmap ceph clusters/{fixed-2 openstack} fast/normal mon_election/connectivity msgr-failures/osd-delay rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/minsize_recovery thrashosds-health workloads/ec-snaps-few-objects-overwrites} time: 00:29:26 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291376/ [7291378] rados/thrash-erasure-code-big/{ceph cluster/{12-osds openstack} mon_election/classic msgr-failures/few objectstore/bluestore-comp-snappy rados recovery-overrides/{default} supported-random-distro$/{ubuntu_20.04} thrashers/fastread thrashosds-health workloads/ec-rados-plugin=lrc-k=4-m=2-l=3} time: 00:14:58 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291378/ [7291380] rados/singleton/{all/divergent_priors mon_election/connectivity msgr-failures/many msgr/async objectstore/bluestore-bitmap rados supported-random-distro$/{rhel_8}} time: 00:15:04 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291380/ [7291382] rados/cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:15:26 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291382/ [7291384] rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-low-osd-mem-target rados recovery-overrides/{more-partial-recovery} supported-random-distro$/{ubuntu_20.04} thrashers/default thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} time: 00:14:56 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291384/ [7291386] rados/thrash/{0-size-min-size-overrides/3-size-2-min-size 1-pg-log-overrides/short_pg_log 2-recovery-overrides/{more-active-recovery} 3-scrub-overrides/{default} backoff/peering_and_degraded ceph clusters/{fixed-2 openstack} crc-failures/default d-balancer/on mon_election/connectivity msgr-failures/fastclose msgr/async objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_latest} thrashers/pggrow thrashosds-health workloads/radosbench} time: 00:58:32 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291386/ [7291388] rados/singleton-nomsgr/{all/lazy_omap_stats_output mon_election/connectivity rados supported-random-distro$/{centos_8}} time: 00:12:36 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291388/ [7291390] rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/none mon_election/classic msgr-failures/few msgr/async objectstore/bluestore-low-osd-mem-target rados tasks/rados_api_tests validater/valgrind} time: 00:51:29 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291390/ [7291392] rados/cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:14:03 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291392/ [7291394] rados/singleton/{all/divergent_priors2 mon_election/classic msgr-failures/none msgr/async-v1only objectstore/bluestore-comp-lz4 rados supported-random-distro$/{ubuntu_20.04}} time: 00:10:54 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291394/ [7291396] rados/perf/{ceph mon_election/connectivity objectstore/bluestore-basic-min-osd-mem-target openstack scheduler/dmclock_default_shards settings/optimized ubuntu_latest workloads/fio_4M_rand_rw} time: 00:13:14 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291396/ [7291398] rados/standalone/{supported-random-distro$/{ubuntu_20.04} workloads/osd} time: 02:41:58 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291398/ [7291400] rados/basic/{ceph clusters/{fixed-2 openstack} mon_election/classic msgr-failures/many msgr/async-v1only objectstore/bluestore-stupid rados supported-random-distro$/{ubuntu_20.04} tasks/libcephsqlite} time: 00:17:17 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291400/ [7291402] rados/singleton-nomsgr/{all/librados_hello_world mon_election/classic rados supported-random-distro$/{rhel_8}} time: 00:19:41 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291402/ [7291404] rados/thrash/{0-size-min-size-overrides/2-size-2-min-size 1-pg-log-overrides/normal_pg_log 2-recovery-overrides/{more-async-recovery} 3-scrub-overrides/{default} backoff/normal ceph clusters/{fixed-2 openstack} crc-failures/bad_map_crc_failure d-balancer/crush-compat mon_election/classic msgr-failures/few msgr/async-v1only objectstore/bluestore-bitmap rados supported-random-distro$/{ubuntu_20.04} thrashers/careful thrashosds-health workloads/redirect} time: 00:19:14 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291404/ [7291406] rados/cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:12 info: http://pulpito.ceph.com/yuriw-2023-05-30_21:40:27-rados-wip-yuri4-testing-2023-05-30-0825-reef-distro-default-smithi/7291406/