2024-01-26T08:07:24.004 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-26T08:07:24.004 INFO:teuthology.results:subset: '1/8' 2024-01-26T08:07:24.004 INFO:teuthology.results:seed: '7364' 2024-01-26T10:21:18.603 INFO:root:teuthology version: 0.0.1.dev249+gcd45576 2024-01-26T10:21:18.636 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2024-01-26T11:14:22.918 INFO:teuthology.results:Tests finished! gathering results... 2024-01-26T11:14:28.846 INFO:scrape:Found 101 jobs 2024-01-26T11:14:30.397 INFO:scrape:Found 3 distinct failure reasons 2024-01-26T11:14:30.397 INFO:scrape:Failure: '2024-01-26T10:05:53.360896+00:00 smithi073 kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks: ' in syslog 2024-01-26T11:14:30.398 INFO:scrape:1 jobs: ['7533996'] 2024-01-26T11:14:30.398 INFO:scrape:suites: ['1-bootstrap/16.2.5', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2024-01-26T11:14:30.398 INFO:scrape: 2024-01-26T11:14:30.398 INFO:scrape:Failure: Command failed on smithi089 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8f510eb24a51fa6e2c9ffea04684fa45f25cf2c4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e0d1edb2-bc32-11ee-95b2-87774f69a715 -- ceph mon dump -f json' 2024-01-26T11:14:30.398 INFO:scrape:2 jobs: ['7534025', '7534019'] 2024-01-26T11:14:30.398 INFO:scrape:suites intersection: ['1-start'] 2024-01-26T11:14:30.398 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/rgw', '2-thrash', '3-final}', '3-tasks/rados_api_tests', 'fixed-2', 'msgr/async-v1only', 'orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0', 'root}'] 2024-01-26T11:14:30.398 INFO:scrape: 2024-01-26T11:14:30.398 INFO:scrape:Failure: Test failure: test_non_existent_cluster (tasks.cephfs.test_nfs.TestNFS) 2024-01-26T11:14:30.398 INFO:scrape:1 jobs: ['7533963'] 2024-01-26T11:14:30.398 INFO:scrape:suites: ['orch:cephadm/nfs/{cluster/{1-node}', 'overrides/ignorelist_health', 'supported-random-distros$/{ubuntu_latest}', 'tasks/nfs}'] 2024-01-26T11:14:30.399 INFO:scrape: 2024-01-26T11:14:30.402 INFO:teuthology.results:Sending results to gabrioux@ibm.com: Test Run: gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi ================================================================= info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/ failed: 4 dead: 0 running: 0 waiting: 0 queued: 0 passed: 97 Fail ================================================================= [7533963] orch:cephadm/nfs/{cluster/{1-node} overrides/ignorelist_health supported-random-distros$/{ubuntu_latest} tasks/nfs} ----------------------------------------------------------------- time: 00:29:22 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533963/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533963/ sentry: https://sentry.ceph.com/organizations/ceph/?query=707c583cc58e4c03b2d7eae2fde3bd47 Test failure: test_non_existent_cluster (tasks.cephfs.test_nfs.TestNFS) [7533996] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:48:05 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533996/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533996/ '2024-01-26T10:05:53.360896+00:00 smithi073 kernel: rcu: INFO: rcu_sched detected stalls on CPUs/tasks: ' in syslog [7534019] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw 3-final} ----------------------------------------------------------------- time: 00:14:53 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534019/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534019/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f036aaf0e9a94e5589397d20a6992d90 Command failed on smithi038 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:8f510eb24a51fa6e2c9ffea04684fa45f25cf2c4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 5aafe1d0-bc32-11ee-95b2-87774f69a715 -- ceph orch ls -f json' [7534025] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} ----------------------------------------------------------------- time: 00:20:09 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534025/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534025/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3268af10e2c04004ac838a190c95ffa9 Command failed on smithi089 with status 127: 'sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:8f510eb24a51fa6e2c9ffea04684fa45f25cf2c4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e0d1edb2-bc32-11ee-95b2-87774f69a715 -- ceph mon dump -f json' Pass ================================================================= [7533958] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:14:44 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533958/ [7533959] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:27:12 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533959/ [7533960] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:15:19 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533960/ [7533961] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:36 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533961/ [7533962] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:31:42 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533962/ [7533964] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 2-node-mgr agent/off orchestrator_cli} time: 00:19:44 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533964/ [7533965] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} time: 00:39:47 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533965/ [7533966] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:12:21 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533966/ [7533967] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:25:27 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533967/ [7533968] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli} time: 00:16:45 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533968/ [7533969] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} time: 00:31:19 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533969/ [7533970] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:23:17 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533970/ [7533971] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:27:29 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533971/ [7533972] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:36 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533972/ [7533973] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:42:06 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533973/ [7533974] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:42:50 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533974/ [7533975] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_rgw_multisite} time: 00:27:59 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533975/ [7533976] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:19:32 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533976/ [7533977] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:24:02 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533977/ [7533978] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:04 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533978/ [7533979] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:30:06 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533979/ [7533980] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} time: 00:25:23 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533980/ [7533981] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 01:19:00 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533981/ [7533982] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:53:43 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533982/ [7533983] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:24:11 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533983/ [7533984] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:26:54 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533984/ [7533985] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 01:00:39 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533985/ [7533986] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:46:39 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533986/ [7533987] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_adoption} time: 00:10:17 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533987/ [7533988] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:36:00 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533988/ [7533989] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:24:00 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533989/ [7533990] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_cephadm} time: 00:45:51 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533990/ [7533991] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:19:28 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533991/ [7533992] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} time: 00:39:38 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533992/ [7533993] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} time: 00:33:25 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533993/ [7533994] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_cephadm_repos} time: 00:26:15 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533994/ [7533995] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:59:16 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533995/ [7533997] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:32:51 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533997/ [7533998] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:59:49 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533998/ [7533999] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:34:47 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7533999/ [7534000] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 01:03:13 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534000/ [7534001] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:42:32 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534001/ [7534002] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:20:38 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534002/ [7534003] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:44:51 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534003/ [7534004] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:20:03 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534004/ [7534005] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:18:43 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534005/ [7534006] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} time: 00:31:23 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534006/ [7534007] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:17:25 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534007/ [7534008] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:42:50 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534008/ [7534009] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:26:41 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534009/ [7534010] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:47:59 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534010/ [7534011] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:43:38 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534011/ [7534012] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:26:02 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534012/ [7534013] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:31 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534013/ [7534014] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 2-node-mgr agent/on orchestrator_cli} time: 00:18:55 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534014/ [7534015] orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/rgw 3-final} time: 00:15:35 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534015/ [7534016] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:42:46 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534016/ [7534017] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 00:17:54 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534017/ [7534018] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:30:44 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534018/ [7534020] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:20:09 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534020/ [7534021] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} time: 00:25:59 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534021/ [7534022] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:42 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534022/ [7534023] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:21:12 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534023/ [7534024] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:14:22 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534024/ [7534026] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:35:02 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534026/ [7534027] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} time: 00:17:56 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534027/ [7534028] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:30:11 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534028/ [7534029] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} time: 00:24:57 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534029/ [7534030] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:20:14 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534030/ [7534031] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:05 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534031/ [7534032] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:32:58 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534032/ [7534033] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/defaut 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:34:09 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534033/ [7534034] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} time: 00:30:55 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534034/ [7534035] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:07:27 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534035/ [7534036] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:18:34 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534036/ [7534037] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:19:14 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534037/ [7534038] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:58:26 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534038/ [7534039] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:45:39 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534039/ [7534040] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:16:53 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534040/ [7534041] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:20:11 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534041/ [7534042] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:23:32 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534042/ [7534043] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:21:40 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534043/ [7534044] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_orch_cli} time: 00:16:21 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534044/ [7534045] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} time: 00:26:44 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534045/ [7534046] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:51 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534046/ [7534047] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:52:58 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534047/ [7534048] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:16:37 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534048/ [7534049] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:40:22 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534049/ [7534050] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:26:27 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534050/ [7534051] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:43:31 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534051/ [7534052] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:46:44 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534052/ [7534053] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_rgw_multisite} time: 00:26:58 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534053/ [7534054] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:16:05 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534054/ [7534055] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:28:01 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534055/ [7534056] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:35 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534056/ [7534057] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:23:56 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534057/ [7534058] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} time: 00:37:28 info: https://pulpito.ceph.com/gabrioux-2024-01-26_08:01:26-orch:cephadm-wip-guits-testing-2-2024-01-24-1531-quincy-distro-default-smithi/7534058/