2023-07-27T15:33:41.745 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T15:33:41.746 INFO:teuthology.results:subset: '1/8' 2023-07-27T15:33:41.746 INFO:teuthology.results:seed: '2415' 2023-07-27T17:37:52.995 INFO:root:teuthology version: 0.0.1.dev160+g407880c 2023-07-27T17:37:53.016 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-07-28T05:20:49.191 INFO:teuthology.results:Tests finished! gathering results... 2023-07-28T05:20:58.333 INFO:scrape:Found 111 jobs 2023-07-28T05:20:58.625 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354604/teuthology.log 2023-07-28T05:21:01.182 INFO:scrape:Found 10 distinct failure reasons 2023-07-28T05:21:01.183 INFO:scrape:Failure: reached maximum tries (301) after waiting for 300 seconds 2023-07-28T05:21:01.184 INFO:scrape:1 jobs: ['7354569'] 2023-07-28T05:21:01.184 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-services/jaeger', '3-final}', 'orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0'] 2023-07-28T05:21:01.184 INFO:scrape: 2023-07-28T05:21:01.185 INFO:scrape:Dead: None 2023-07-28T05:21:01.185 INFO:scrape:19 jobs: ['7354604', '7354617', '7354659', '7354577', '7354610', '7354598', '7354595', '7354661', '7354662', '7354653', '7354631', '7354605', '7354618', '7354660', '7354644', '7354599', '7354649', '7354652', '7354590'] 2023-07-28T05:21:01.185 INFO:scrape: 2023-07-28T05:21:01.185 INFO:scrape:Failure: Stale jobs detected, aborting. 2023-07-28T05:21:01.186 INFO:scrape:1 jobs: ['7354581'] 2023-07-28T05:21:01.186 INFO:scrape:suites: ['fixed-2', 'mode/root', 'mon_election/connectivity', 'msgr/async-v2only', 'orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8', 'start', 'tasks/rotate-keys}'] 2023-07-28T05:21:01.186 INFO:scrape: 2023-07-28T05:21:01.187 INFO:scrape:Failure: {"dd: failed to open '/home/ubuntu/.ansible/tmp/ansible-tmp-1690473234.0123048-10153-229378452977024/AnsiballZ_user.py': No such file or directory", 'failed to transfer file to /home/teuthworker/.ansible/tmp/ansible-local-26232qyx3zaaa/tmplj90c2vk /home/ubuntu/.ansible/tmp/ansible-tmp-1690473234.0123048-10153-229378452977024/AnsiballZ_user.py:'} 2023-07-28T05:21:01.187 INFO:scrape:1 jobs: ['7354579'] 2023-07-28T05:21:01.187 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'task/test_nfs}'] 2023-07-28T05:21:01.187 INFO:scrape: 2023-07-28T05:21:01.188 INFO:scrape:Failure: Command failed on smithi172 with status 127: "sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:82c085f827575f079a030cf33416b5e78568774f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f3ed2080-2ca2-11ee-9b35-001a4aab830c -- ceph orch apply prometheus '1;smithi172=a'" 2023-07-28T05:21:01.188 INFO:scrape:1 jobs: ['7354655'] 2023-07-28T05:21:01.188 INFO:scrape:suites: ['0-nvme-loop', 'agent/off', 'fixed-2', 'mon_election/connectivity', 'orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8', 'start}'] 2023-07-28T05:21:01.188 INFO:scrape: 2023-07-28T05:21:01.189 INFO:scrape:Dead: b"TypeError: argument of type 'NoneType' is not iterable" 2023-07-28T05:21:01.189 INFO:scrape:1 jobs: ['7354621'] 2023-07-28T05:21:01.189 INFO:scrape:suites: ['fixed-2', 'mode/packaged', 'mon_election/classic', 'msgr/async-v1only', 'orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0', 'start', 'tasks/rotate-keys}'] 2023-07-28T05:21:01.190 INFO:scrape: 2023-07-28T05:21:01.190 INFO:scrape:Failure: Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) 2023-07-28T05:21:01.190 INFO:scrape:2 jobs: ['7354620', '7354665'] 2023-07-28T05:21:01.190 INFO:scrape:suites intersection: ['agent/on', 'mon_election/connectivity', 'task/test_nfs}'] 2023-07-28T05:21:01.191 INFO:scrape:suites union: ['agent/on', 'mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0', 'task/test_nfs}'] 2023-07-28T05:21:01.191 INFO:scrape: 2023-07-28T05:21:01.191 INFO:scrape:Dead: b'2023-07-28T05:17:18.369 DEBUG:teuthology.exit:Finished running handlers' 2023-07-28T05:21:01.192 INFO:scrape:1 jobs: ['7354657'] 2023-07-28T05:21:01.192 INFO:scrape:suites: ['fixed-2', 'mode/root', 'mon_election/connectivity', 'msgr/async-v2only', 'orch:cephadm/with-work/{0-distro/ubuntu_20.04', 'start', 'tasks/rados_api_tests}'] 2023-07-28T05:21:01.192 INFO:scrape: 2023-07-28T05:21:01.192 INFO:scrape:Failure: SELinux denials found on ubuntu@smithi120.front.sepia.ceph.com: ['type=AVC msg=audit(1690475817.885:20755): avc: denied { ioctl } for pid=112491 comm="iptables" path="/var/lib/containers/storage/overlay/28fd6fa4e60cb361f2aab12c2f7faa1e7bfac6f449769713c26b9aa8e7a47ac7/merged" dev="overlay" ino=2754149 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1690475817.825:20754): avc: denied { ioctl } for pid=112486 comm="iptables" path="/var/lib/containers/storage/overlay/28fd6fa4e60cb361f2aab12c2f7faa1e7bfac6f449769713c26b9aa8e7a47ac7/merged" dev="overlay" ino=2754149 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] 2023-07-28T05:21:01.193 INFO:scrape:1 jobs: ['7354601'] 2023-07-28T05:21:01.193 INFO:scrape:suites: ['agent/on', 'mon_election/connectivity', 'orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8', 'task/test_cephadm}'] 2023-07-28T05:21:01.193 INFO:scrape: 2023-07-28T05:21:01.193 INFO:scrape:Failure: Cannot connect to remote host smithi157 2023-07-28T05:21:01.194 INFO:scrape:1 jobs: ['7354606'] 2023-07-28T05:21:01.194 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-wait}', 'orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0'] 2023-07-28T05:21:01.194 INFO:scrape: 2023-07-28T05:21:01.199 INFO:teuthology.results:Sending results to gabrioux@redhat.com: Test Run: gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/ failed: 8 dead: 21 running: 0 waiting: 0 queued: 0 passed: 82 Fail ================================================================= [7354569] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/jaeger 3-final} ----------------------------------------------------------------- time: 00:42:05 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354569/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354569/ sentry: https://sentry.ceph.com/organizations/ceph/?query=67b88987a888487c88f87e93ca6678da reached maximum tries (301) after waiting for 300 seconds [7354579] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:09:32 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354579/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354579/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3c2f0960c7794aa9a363ca58fd9ed53f {"dd: failed to open '/home/ubuntu/.ansible/tmp/ansible- tmp-1690473234.0123048-10153-229378452977024/AnsiballZ_user.py': No such file or directory", 'failed to transfer file to /home/teuthworker/.ansible/tmp/ansible-local-26232qyx3zaaa/tmplj90c2vk /home/ubuntu/.ansible/tmp/ansible- tmp-1690473234.0123048-10153-229378452977024/AnsiballZ_user.py:'} [7354581] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354581/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354581/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0d6803634c054031a2445db419ac31c4 Stale jobs detected, aborting. [7354601] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_cephadm} ----------------------------------------------------------------- time: 00:26:28 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354601/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354601/ SELinux denials found on ubuntu@smithi120.front.sepia.ceph.com: ['type=AVC msg=audit(1690475817.885:20755): avc: denied { ioctl } for pid=112491 comm="iptables" path="/var/lib/containers/storage/overlay/28fd6fa4e60cb361f 2aab12c2f7faa1e7bfac6f449769713c26b9aa8e7a47ac7/merged" dev="overlay" ino=2754149 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1', 'type=AVC msg=audit(1690475817.825:20754): avc: denied { ioctl } for pid=112486 comm="iptables" path="/var/lib/containers/storage/o verlay/28fd6fa4e60cb361f2aab12c2f7faa1e7bfac6f449769713c26b9aa8e7a47ac7/mer ged" dev="overlay" ino=2754149 scontext=system_u:system_r:iptables_t:s0 tcontext=system_u:object_r:container_file_t:s0:c1022,c1023 tclass=dir permissive=1'] [7354606] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354606/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354606/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f260a1f0744d4eea8af87c44d31b6a32 Cannot connect to remote host smithi157 [7354620] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:21:30 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354620/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354620/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2113b9d7ab9043bd8623bc7326548dfd Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) [7354655] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:19:18 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354655/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354655/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1541a6cc69aa45a98948e1f8bd109ad6 Command failed on smithi172 with status 127: "sudo /home/ubuntu/cephtest/cephadm --image quay-quay- quay.apps.os.sepia.ceph.com/ceph- ci/ceph:82c085f827575f079a030cf33416b5e78568774f shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f3ed2080-2ca2-11ee-9b35-001a4aab830c -- ceph orch apply prometheus '1;smithi172=a'" [7354665] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 01:26:55 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354665/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354665/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a216c34756eb4b8785133f1a3aa18a0c Test failure: test_cluster_info (tasks.cephfs.test_nfs.TestNFS) Dead ================================================================= [7354577] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354577/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354577/ Error reimaging machines: 500 Server Error: Internal Server Error for url: http://fog.front.sepia.ceph.com/fog/host/264/task [7354590] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354590/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354590/ Error reimaging machines: 'ssh_keyscan smithi159.front.sepia.ceph.com' reached maximum tries (16) after waiting for 120 seconds [7354595] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354595/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354595/ Error reimaging machines: Failed to power on smithi046 [7354598] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354598/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354598/ Error reimaging machines: Failed to power on smithi157 [7354599] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_adoption} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354599/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354599/ Error reimaging machines: Failed to power on smithi161 [7354604] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354604/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354604/ Error reimaging machines: Failed to power on smithi046 [7354605] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354605/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354605/ Error reimaging machines: Expected smithi157's OS to be ubuntu 20.04 but found rhel 8.6 [7354610] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_extra_daemon_features} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354610/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354610/ Error reimaging machines: Failed to power on smithi046 [7354617] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354617/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354617/ Error reimaging machines: Expected smithi157's OS to be centos 8 but found rhel 8.6 [7354618] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354618/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354618/ Error reimaging machines: Failed to power on smithi077 [7354621] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354621/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354621/ [7354631] orch:cephadm/orchestrator_cli/{0-random-distro$/{centos_8.stream_container_tools_crun} 2-node-mgr agent/on orchestrator_cli} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354631/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354631/ Error reimaging machines: Failed to power on smithi190 [7354644] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354644/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354644/ Error reimaging machines: Failed to power on smithi084 [7354649] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354649/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354649/ Error reimaging machines: Failed to power on smithi064 [7354652] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354652/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354652/ Error reimaging machines: Failed to power on smithi008 [7354653] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354653/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354653/ Error reimaging machines: Expected smithi079's OS to be centos 8 but found ubuntu 20.04 [7354657] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354657/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354657/ hit max job timeout [7354659] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354659/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354659/ Error reimaging machines: Expected smithi123's OS to be centos 8 but found ubuntu 20.04 [7354660] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354660/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354660/ Error reimaging machines: Failed to power on smithi148 [7354661] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354661/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354661/ Error reimaging machines: Expected smithi116's OS to be ubuntu 20.04 but found rhel 8.6 [7354662] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/jaeger 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354662/ log: http://qa-proxy.ceph.com/teuthology/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354662/ Error reimaging machines: Expected smithi098's OS to be centos 8 but found rhel 8.6 Pass ================================================================= [7354570] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:22:10 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354570/ [7354571] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:36:37 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354571/ [7354572] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:33:39 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354572/ [7354573] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:39:18 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354573/ [7354574] orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/off orchestrator_cli} time: 00:14:12 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354574/ [7354575] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} workloads/cephadm_iscsi} time: 00:44:30 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354575/ [7354576] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/basic 3-final} time: 00:13:56 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354576/ [7354578] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:46:45 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354578/ [7354580] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:17:58 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354580/ [7354582] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_orch_cli} time: 00:20:12 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354582/ [7354583] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:58 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354583/ [7354584] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} time: 00:21:59 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354584/ [7354585] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:42:53 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354585/ [7354586] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:49:16 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354586/ [7354587] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:36:43 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354587/ [7354588] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:29:23 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354588/ [7354589] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:16:54 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354589/ [7354591] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_rgw_multisite} time: 00:27:29 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354591/ [7354592] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:35:58 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354592/ [7354593] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} time: 00:34:48 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354593/ [7354594] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:28:37 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354594/ [7354596] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:26:13 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354596/ [7354597] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 01:10:16 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354597/ [7354600] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:28:03 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354600/ [7354602] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:31:47 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354602/ [7354603] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:37:48 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354603/ [7354607] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:20:49 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354607/ [7354608] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_cephadm_repos} time: 00:07:38 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354608/ [7354609] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} time: 00:43:55 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354609/ [7354611] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 00:41:01 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354611/ [7354612] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:16:10 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354612/ [7354613] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} time: 00:21:16 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354613/ [7354614] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:52:44 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354614/ [7354615] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 01:07:25 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354615/ [7354616] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:21:26 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354616/ [7354619] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:16:40 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354619/ [7354622] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:22:12 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354622/ [7354623] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:39:58 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354623/ [7354624] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_orch_cli} time: 00:21:28 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354624/ [7354625] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:39:54 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354625/ [7354626] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 02:02:24 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354626/ [7354627] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_api_tests} time: 00:41:57 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354627/ [7354628] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:39:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354628/ [7354629] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:25:44 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354629/ [7354630] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 01:45:16 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354630/ [7354632] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:28:17 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354632/ [7354633] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:36:30 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354633/ [7354634] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 01:08:44 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354634/ [7354635] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 00:32:16 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354635/ [7354636] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_rgw_multisite} time: 01:34:58 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354636/ [7354637] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:19:04 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354637/ [7354638] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} time: 01:52:58 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354638/ [7354639] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 01:37:14 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354639/ [7354640] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:22 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354640/ [7354641] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:19:59 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354641/ [7354642] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:17:10 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354642/ [7354643] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:49:43 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354643/ [7354645] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_adoption} time: 01:27:44 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354645/ [7354646] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 01:55:34 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354646/ [7354647] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm} time: 00:25:45 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354647/ [7354648] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:33:29 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354648/ [7354650] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} time: 01:01:17 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354650/ [7354651] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_cephadm_repos} time: 00:12:01 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354651/ [7354654] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_extra_daemon_features} time: 00:24:25 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354654/ [7354656] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:54:52 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354656/ [7354658] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 01:53:24 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354658/ [7354663] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 01:34:11 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354663/ [7354664] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_python} time: 01:06:45 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354664/ [7354666] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:21:43 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354666/ [7354667] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rotate-keys} time: 01:44:27 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354667/ [7354668] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_orch_cli} time: 00:26:17 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354668/ [7354669] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:26:15 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354669/ [7354670] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} time: 00:20:42 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354670/ [7354671] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:44:00 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354671/ [7354672] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:49:14 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354672/ [7354673] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 01:45:37 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354673/ [7354674] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:33:58 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354674/ [7354675] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:31:11 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354675/ [7354676] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:32:12 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354676/ [7354677] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:21:26 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354677/ [7354678] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/classic task/test_rgw_multisite} time: 00:18:27 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354678/ [7354679] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rotate-keys} time: 00:38:03 info: http://pulpito.ceph.com/gabrioux-2023-07-27_15:32:12-orch:cephadm-wip-guits-testing-5-2023-07-26-0921-distro-default-smithi/7354679/