2023-04-25T14:35:26.387 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-25T14:35:26.388 INFO:teuthology.results:subset: '255/1024' 2023-04-25T14:35:26.388 INFO:teuthology.results:seed: '4920' 2023-04-25T15:02:42.600 INFO:root:teuthology version: 0.0.1.dev96+g8d156ae 2023-04-25T15:02:42.604 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-04-26T03:03:40.513 WARNING:teuthology.results:test(s) did not finish before timeout of 43200 seconds 2023-04-26T03:03:42.443 INFO:scrape:Found 25 jobs 2023-04-26T03:03:47.273 INFO:scrape:Found 4 distinct failure reasons 2023-04-26T03:03:47.274 INFO:scrape:Failure: Command failed on smithi033 with status 1: 'sudo yum upgrade -y linux-firmware' 2023-04-26T03:03:47.274 INFO:scrape:2 jobs: ['7252026', '7252019'] 2023-04-26T03:03:47.275 INFO:scrape:suites intersection: ['1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2023-04-26T03:03:47.275 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2023-04-26T03:03:47.275 INFO:scrape: 2023-04-26T03:03:47.276 INFO:scrape:Failure: Test failure: test_nfs_export_creation_at_symlink (tasks.cephfs.test_nfs.TestNFS) 2023-04-26T03:03:47.276 INFO:scrape:2 jobs: ['7252024', '7252015'] 2023-04-26T03:03:47.276 INFO:scrape:suites intersection: ['agent/off', 'task/test_nfs}'] 2023-04-26T03:03:47.277 INFO:scrape:suites union: ['agent/off', 'mon_election/classic', 'mon_election/connectivity', 'orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun', 'orch/cephadm/workunits/{0-distro/ubuntu_20.04', 'task/test_nfs}'] 2023-04-26T03:03:47.277 INFO:scrape: 2023-04-26T03:03:47.277 INFO:scrape:Failure: Command failed on smithi002 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" 2023-04-26T03:03:47.277 INFO:scrape:18 jobs: ['7252017', '7252011', '7252007', '7252010', '7252025', '7252029', '7252016', '7252027', '7252031', '7252018', '7252020', '7252022', '7252009', '7252030', '7252013', '7252008', '7252021', '7252028'] 2023-04-26T03:03:47.278 INFO:scrape:suites intersection: ['0-nvme-loop', '1-start', '3-final}'] 2023-04-26T03:03:47.278 INFO:scrape:suites union: ['0-nvme-loop', '1-start', '2-services/nfs', '2-services/nfs-ingress', '2-services/nfs-ingress-rgw-bucket', '2-services/nfs-ingress-rgw-user', '2-services/nfs-ingress2', '2-services/nfs2', '3-final}', 'orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools', 'orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0', 'orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8', 'orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04'] 2023-04-26T03:03:47.278 INFO:scrape: 2023-04-26T03:03:47.279 INFO:scrape:Dead: b'2023-04-26T02:51:39.303 DEBUG:teuthology.exit:Finished running handlers' 2023-04-26T03:03:47.279 INFO:scrape:3 jobs: ['7252014', '7252012', '7252023'] 2023-04-26T03:03:47.279 INFO:scrape:suites intersection: ['1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2023-04-26T03:03:47.280 INFO:scrape:suites union: ['1-bootstrap/16.2.0', '1-bootstrap/16.2.4', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2023-04-26T03:03:47.280 INFO:scrape: 2023-04-26T03:03:47.281 INFO:teuthology.results:Sending results to vshankar@redhat.com: Test Run: vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi ================================================================= info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/ failed: 22 dead: 2 running: 1 waiting: 0 queued: 0 passed: 0 Fail ================================================================= [7252007] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:07:08 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252007/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252007/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3b4b9bf1b3ce4f2bbca69159bbe1e835 Command failed on smithi149 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252008] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:41:29 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252008/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252008/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2cd204e8ef4d4a0a9048483ca8efe394 Command failed on smithi042 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252009] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:43:30 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252009/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252009/ sentry: https://sentry.ceph.com/organizations/ceph/?query=6ce1467c3fe646f08e7e1ec6af29b5aa Command failed on smithi029 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252010] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:15:35 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252010/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252010/ sentry: https://sentry.ceph.com/organizations/ceph/?query=61ffaa58149343fda6bdf4b59064a0d6 Command failed on smithi031 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252011] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:10:41 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252011/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252011/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f273217cc72a4dec98f9cd14a03a858a Command failed on smithi123 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252013] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:08:19 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252013/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252013/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d9d7bafde165423b9770ee250a7d2da1 Command failed on smithi073 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252015] orch/cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/off mon_election/classic task/test_nfs} ----------------------------------------------------------------- time: 01:46:20 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252015/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252015/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3a51b194ba7a48ab8445e38486e6332d Test failure: test_nfs_export_creation_at_symlink (tasks.cephfs.test_nfs.TestNFS) [7252016] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:11:10 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252016/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252016/ sentry: https://sentry.ceph.com/organizations/ceph/?query=051e03505a6d4e298c9767f5be3b8720 Command failed on smithi088 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252017] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:10:57 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252017/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252017/ sentry: https://sentry.ceph.com/organizations/ceph/?query=63bc4a61dabd459e9377dcf2db0a4ca3 Command failed on smithi002 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252018] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:07:45 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252018/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252018/ sentry: https://sentry.ceph.com/organizations/ceph/?query=f8ae35fe87be4e5c9dd1fa9d5546535b Command failed on smithi084 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252019] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252019/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252019/ sentry: https://sentry.ceph.com/organizations/ceph/?query=25c2c49abd4f4bd792e64b4c7d4b1b32 Command failed on smithi179 with status 1: 'sudo yum upgrade -y linux- firmware' [7252020] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:41:18 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252020/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252020/ sentry: https://sentry.ceph.com/organizations/ceph/?query=d745be134a37429d99601f80919ab0dc Command failed on smithi112 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252021] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:41:36 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252021/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252021/ sentry: https://sentry.ceph.com/organizations/ceph/?query=dc8980c239624aa583bad870f7c17d43 Command failed on smithi066 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252022] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:11:26 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252022/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252022/ sentry: https://sentry.ceph.com/organizations/ceph/?query=5c5cbf9045e14ddfb466290e9c359054 Command failed on smithi046 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252024] orch/cephadm/workunits/{0-distro/ubuntu_20.04 agent/off mon_election/connectivity task/test_nfs} ----------------------------------------------------------------- time: 00:31:22 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252024/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252024/ sentry: https://sentry.ceph.com/organizations/ceph/?query=06bbef5df3a8499592e9c1f1158dd85a Test failure: test_nfs_export_creation_at_symlink (tasks.cephfs.test_nfs.TestNFS) [7252025] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} ----------------------------------------------------------------- time: 00:41:34 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252025/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252025/ sentry: https://sentry.ceph.com/organizations/ceph/?query=012ad0908b94496ba2ecf191ddff9ff8 Command failed on smithi149 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252026] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252026/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252026/ sentry: https://sentry.ceph.com/organizations/ceph/?query=0980a3c33e294c60861e2c3ae50f7f75 Command failed on smithi033 with status 1: 'sudo yum upgrade -y linux- firmware' [7252027] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} ----------------------------------------------------------------- time: 00:41:48 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252027/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252027/ sentry: https://sentry.ceph.com/organizations/ceph/?query=b231d19720434ff9adb9196e9b4b1a57 Command failed on smithi107 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252028] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} ----------------------------------------------------------------- time: 00:15:43 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252028/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252028/ sentry: https://sentry.ceph.com/organizations/ceph/?query=3d592c00e0b74460b20db7265804cd8a Command failed on smithi006 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252029] orch/cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} ----------------------------------------------------------------- time: 00:14:55 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252029/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252029/ sentry: https://sentry.ceph.com/organizations/ceph/?query=1e28ab51f31b4167a3be7a7a0b982a24 Command failed on smithi096 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252030] orch/cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs 3-final} ----------------------------------------------------------------- time: 00:07:08 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252030/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252030/ sentry: https://sentry.ceph.com/organizations/ceph/?query=2f1a3bee08b7440ebeed0a887006c732 Command failed on smithi123 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" [7252031] orch/cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} ----------------------------------------------------------------- time: 00:41:21 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252031/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252031/ sentry: https://sentry.ceph.com/organizations/ceph/?query=158a225ae3774af487fb5bd429c1a666 Command failed on smithi073 with status 1: "grep '^nvme_loop' /proc/modules || sudo modprobe nvme_loop && sudo mkdir -p /sys/kernel/config/nvmet/hosts/hostnqn && sudo mkdir -p /sys/kernel/config/nvmet/ports/1 && echo loop | sudo tee /sys/kernel/config/nvmet/ports/1/addr_trtype" Dead ================================================================= [7252012] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252012/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252012/ hit max job timeout [7252014] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252014/ log: http://qa-proxy.ceph.com/teuthology/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252014/ hit max job timeout Running ================================================================= [7252023] orch/cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} info: http://pulpito.ceph.com/vshankar-2023-04-25_14:33:31-orch-wip-vshankar-testing-20230420.132447-testing-default-smithi/7252023/