2023-11-03T15:35:09.290 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-03T15:35:09.291 INFO:teuthology.results:subset: '1/15' 2023-11-03T15:35:09.291 INFO:teuthology.results:seed: '7852' 2023-11-05T06:50:17.405 INFO:root:teuthology version: 0.0.1.dev203+g6899cd2 2023-11-05T06:50:17.439 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2023-11-05T07:38:21.496 INFO:teuthology.results:Tests finished! gathering results... 2023-11-05T07:38:27.272 INFO:scrape:Found 88 jobs 2023-11-05T07:38:27.273 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444486/teuthology.log 2023-11-05T07:38:27.727 INFO:scrape:Found 3 distinct failure reasons 2023-11-05T07:38:27.728 INFO:scrape:Dead: None 2023-11-05T07:38:27.729 INFO:scrape:1 jobs: ['7444486'] 2023-11-05T07:38:27.729 INFO:scrape: 2023-11-05T07:38:27.730 INFO:scrape:Dead: b'2023-11-04T04:30:43.869 DEBUG:teuthology.exit:Finished running handlers' 2023-11-05T07:38:27.730 INFO:scrape:1 jobs: ['7444447'] 2023-11-05T07:38:27.730 INFO:scrape:suites: ['1-start', '2-thrash', '3-tasks/snaps-few-objects', 'fixed-2', 'msgr/async', 'orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun', 'root}'] 2023-11-05T07:38:27.731 INFO:scrape: 2023-11-05T07:38:27.731 INFO:scrape:Failure: Command failed on smithi069 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 995acebc-7a61-11ee-8db9-212e2dc638e7 -e sha1=d5bf8a36f8bfe15015612ffbe24633090fa00c76 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' 2023-11-05T07:38:27.732 INFO:scrape:1 jobs: ['7444356'] 2023-11-05T07:38:27.732 INFO:scrape:suites: ['1-bootstrap/17.2.0', '1-start', '2-nfs', '3-upgrade-with-workload', '4-final}', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools'] 2023-11-05T07:38:27.732 INFO:scrape: 2023-11-05T07:38:27.736 INFO:teuthology.results:Sending results to jmulligan@redhat.com: Test Run: phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/ failed: 1 dead: 2 running: 0 waiting: 0 queued: 0 passed: 85 Fail ================================================================= [7444356] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/17.2.0 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:30:34 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444356/ log: http://qa-proxy.ceph.com/teuthology/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444356/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7112b82fe8514faabbf1210af20e068f Command failed on smithi069 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v17.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 995acebc-7a61-11ee-8db9-212e2dc638e7 -e sha1=d5bf8a36f8bfe15015612ffbe24633090fa00c76 -- bash -c \'ceph versions | jq -e \'"\'"\'.overall | length == 1\'"\'"\'\'' Dead ================================================================= [7444447] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444447/ log: http://qa-proxy.ceph.com/teuthology/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444447/ hit max job timeout [7444486] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/client-keyring 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444486/ log: http://qa-proxy.ceph.com/teuthology/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444486/ Error reimaging machines: reached maximum tries (101) after waiting for 600 seconds Pass ================================================================= [7444350] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:15:53 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444350/ [7444351] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:46:11 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444351/ [7444352] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:11:51 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444352/ [7444353] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} time: 00:16:28 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444353/ [7444358] orch:cephadm/nfs/{cluster/{1-node} overrides/ignorelist_health supported-random-distros$/{centos_latest} tasks/nfs} time: 00:24:56 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444358/ [7444361] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0} 2-node-mgr agent/off orchestrator_cli} time: 00:21:47 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444361/ [7444363] orch:cephadm/rbd_iscsi/{0-single-container-host base/install cluster/{fixed-3 openstack} conf/{disable-pool-app} workloads/cephadm_iscsi} time: 00:38:23 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444363/ [7444366] orch:cephadm/smoke-singlehost/{0-random-distro$/{rhel_8.6_container_tools_rhel8} 1-start 2-services/basic 3-final} time: 00:15:05 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444366/ [7444368] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:13:08 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444368/ [7444371] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:41:41 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444371/ [7444373] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:43:38 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444373/ [7444376] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:21:20 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444376/ [7444378] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:15:20 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444378/ [7444381] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:21:00 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444381/ [7444383] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_extra_daemon_features} time: 00:21:53 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444383/ [7444386] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:18:39 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444386/ [7444388] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:28:09 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444388/ [7444391] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:29:57 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444391/ [7444393] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:23:10 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444393/ [7444395] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 01:14:29 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444395/ [7444398] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:16:26 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444398/ [7444400] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rotate-keys} time: 00:37:58 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444400/ [7444403] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:17:43 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444403/ [7444405] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:43:32 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444405/ [7444408] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:18:14 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444408/ [7444410] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:18:17 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444410/ [7444413] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_orch_cli} time: 00:16:23 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444413/ [7444415] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-keepalive-only 3-final} time: 00:15:11 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444415/ [7444417] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:14:16 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444417/ [7444420] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:30:44 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444420/ [7444422] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:36:55 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444422/ [7444425] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:19:18 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444425/ [7444427] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_orch_cli_mon} time: 00:26:19 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444427/ [7444430] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:31:03 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444430/ [7444432] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:17:01 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444432/ [7444435] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:23:55 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444435/ [7444437] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rmdir-reactivate} time: 00:16:56 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444437/ [7444440] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:25:38 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444440/ [7444442] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_rgw_multisite} time: 00:22:03 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444442/ [7444445] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nvmeof 3-final} time: 00:19:40 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444445/ [7444450] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:19:11 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444450/ [7444452] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-upgrade/staggered 4-wait 5-upgrade-ls agent/on mon_election/classic} time: 00:46:41 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444452/ [7444454] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} time: 00:32:49 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444454/ [7444457] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_set_mon_crush_locations} time: 00:26:11 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444457/ [7444459] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:16:08 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444459/ [7444462] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:31:32 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444462/ [7444464] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.6_container_tools_3.0} 2-node-mgr agent/on orchestrator_cli} time: 00:25:28 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444464/ [7444467] orch:cephadm/osds/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-ops/repave-all} time: 00:18:58 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444467/ [7444469] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:16:15 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444469/ [7444471] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:20:49 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444471/ [7444474] orch:cephadm/smoke-singlehost/{0-random-distro$/{centos_8.stream_container_tools_crun} 1-start 2-services/rgw 3-final} time: 00:13:46 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444474/ [7444476] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:14:29 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444476/ [7444479] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:43:08 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444479/ [7444481] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:42:47 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444481/ [7444484] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/off mon_election/classic task/test_adoption} time: 00:17:42 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444484/ [7444489] orch:cephadm/workunits/{0-distro/ubuntu_20.04 agent/on mon_election/connectivity task/test_ca_signed_key} time: 00:16:31 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444489/ [7444491] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:31:49 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444491/ [7444494] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:20:09 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444494/ [7444496] orch:cephadm/thrash/{0-distro/rhel_8.6_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:51:00 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444496/ [7444498] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/jaeger 3-final} time: 00:16:14 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444498/ [7444501] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/connectivity} time: 00:37:53 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444501/ [7444503] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:27 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444503/ [7444506] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:16:11 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444506/ [7444508] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools agent/off mon_election/classic task/test_cephadm} time: 00:18:12 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444508/ [7444511] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rotate-keys} time: 00:26:32 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444511/ [7444513] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/no overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-inline/yes 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:30:43 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444513/ [7444516] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final} time: 00:14:33 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444516/ [7444519] orch:cephadm/workunits/{0-distro/centos_8.stream_container_tools_crun agent/on mon_election/connectivity task/test_cephadm_repos} time: 00:09:34 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444519/ [7444522] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-haproxy-proto 3-final} time: 00:19:47 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444522/ [7444527] orch:cephadm/smoke-small/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/off fixed-2 mon_election/connectivity start} time: 00:14:18 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444527/ [7444529] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:43:13 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444529/ [7444532] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} time: 00:36:39 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444532/ [7444534] orch:cephadm/smoke-roleless/{0-distro/rhel_8.6_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:18:56 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444534/ [7444537] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:14:29 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444537/ [7444539] orch:cephadm/smoke/{0-distro/rhel_8.6_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/classic start} time: 00:20:03 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444539/ [7444542] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_3.0 agent/off mon_election/classic task/test_extra_daemon_features} time: 00:20:44 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444542/ [7444544] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:22:04 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444544/ [7444547] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_python} time: 00:28:50 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444547/ [7444549] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/quincy 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-inline/no 4-verify} 2-client 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} time: 00:32:06 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444549/ [7444552] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:17:46 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444552/ [7444554] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:38:16 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444554/ [7444557] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-upgrade/simple 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:28:13 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444557/ [7444559] orch:cephadm/workunits/{0-distro/rhel_8.6_container_tools_rhel8 agent/on mon_election/connectivity task/test_iscsi_container/{centos_8.stream_container_tools test_iscsi_container}} time: 00:16:46 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444559/ [7444561] orch:cephadm/with-work/{0-distro/rhel_8.6_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rotate-keys} time: 00:34:57 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444561/ [7444564] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:17:17 info: http://pulpito.ceph.com/phlogistonjohn-2023-11-02_14:19:20-orch:cephadm-wip-phlogistonjohn-testing-2023-11-01-2015-distro-default-smithi/7444564/