2021-11-29T16:03:08.450 INFO:root:teuthology version: 1.1.0-a0bc3188 2021-11-29T16:03:08.451 INFO:teuthology.results:subset: '0/6' 2021-11-29T16:03:08.452 INFO:teuthology.results:seed: '9427' 2021-11-29T17:00:05.143 INFO:root:teuthology version: 1.1.0-a0bc3188 2021-11-29T17:00:05.174 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2021-11-29T18:03:10.138 INFO:teuthology.results:Tests finished! gathering results... 2021-11-29T18:03:16.488 INFO:scrape:Found 100 jobs 2021-11-29T18:03:16.489 WARNING:scrape:Missing teuthology log /home/teuthworker/archive/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533716/teuthology.log 2021-11-29T18:03:17.414 INFO:scrape:Found 3 distinct failure reasons 2021-11-29T18:03:17.415 INFO:scrape:Dead: None 2021-11-29T18:03:17.416 INFO:scrape:4 jobs: ['6533716', '6533700', '6533752', '6533734'] 2021-11-29T18:03:17.416 INFO:scrape: 2021-11-29T18:03:17.417 INFO:scrape:Failure: Command failed on smithi109 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7943c784-5130-11ec-8c2d-001a4aab830c -- bash -c 'ceph config get mgr mgr/cephadm/migration_current | grep 3'" 2021-11-29T18:03:17.417 INFO:scrape:4 jobs: ['6533713', '6533747', '6533703', '6533675'] 2021-11-29T18:03:17.417 INFO:scrape:suites intersection: ['3-upgrade-with-workload'] 2021-11-29T18:03:17.418 INFO:scrape:suites union: ['1-bootstrap/16.2.4', '1-bootstrap/16.2.5', '1-bootstrap/octopus', '1-ranks/2', '1-start', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '2-nfs', '3-upgrade-with-workload', '3-verify}', '4-final}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap', 'orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools', 'osd}', 'overrides/{pg-warn', 'roles', 'tasks/{0-from/pacific', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2021-11-29T18:03:17.418 INFO:scrape: 2021-11-29T18:03:17.419 INFO:scrape:Failure: Command failed on smithi053 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:caa4f4125affaece3c1649c79f1b56fa72f9867e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3aefb7fe-5130-11ec-8c2d-001a4aab830c -- ceph osd stat -f json' 2021-11-29T18:03:17.419 INFO:scrape:1 jobs: ['6533698'] 2021-11-29T18:03:17.420 INFO:scrape:suites: ['0-nvme-loop', '1-start', '2-ops/rm-zap-flag}', 'orch:cephadm/osds/{0-distro/centos_8.stream_container_tools'] 2021-11-29T18:03:17.420 INFO:scrape: 2021-11-29T18:03:17.423 INFO:teuthology.results:Sending results to sewagner@redhat.com: Test Run: swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/ failed: 5 dead: 4 running: 0 waiting: 0 queued: 0 passed: 91 Fail ================================================================= [6533675] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.4 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:28:38 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533675/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533675/ sentry: https://sentry.ceph.com/organizations/ceph/?query=a6f47efc6dd544428df39aea8895f104 Command failed on smithi026 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 65fb89d2-5130-11ec-8c2d-001a4aab830c -- bash -c 'ceph config get mgr mgr/cephadm/migration_current | grep 3'" [6533698] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-flag} ----------------------------------------------------------------- time: 00:12:29 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533698/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533698/ sentry: https://sentry.ceph.com/organizations/ceph/?query=062c0465cc7d423d996faad6d4eb0a87 Command failed on smithi053 with status 1: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph- ci/ceph:caa4f4125affaece3c1649c79f1b56fa72f9867e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3aefb7fe-5130-11ec-8c2d-001a4aab830c -- ceph osd stat -f json' [6533703] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:17:35 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533703/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533703/ sentry: https://sentry.ceph.com/organizations/ceph/?query=34e992e8e2b14d99bc4ed4632b0beea8 Command failed on smithi149 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/daemon-base:latest- pacific shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid c235026e-5130-11ec-8c2d-001a4aab830c -- ceph orch daemon add osd smithi149:vg_nvme/lv_4' [6533713] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/16.2.5 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:26:46 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533713/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533713/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4602cabd44fb4078bc868fb918778971 Command failed on smithi109 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.5 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 7943c784-5130-11ec-8c2d-001a4aab830c -- bash -c 'ceph config get mgr mgr/cephadm/migration_current | grep 3'" [6533747] orch:cephadm/mgr-nfs-upgrade/{0-centos_8.stream_container_tools 1-bootstrap/octopus 1-start 2-nfs 3-upgrade-with-workload 4-final} ----------------------------------------------------------------- time: 00:35:13 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533747/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533747/ sentry: https://sentry.ceph.com/organizations/ceph/?query=4a8d7877f8574d65a0d99553dd05b65f Command failed on smithi110 with status 1: "sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v15 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c7da032-5136-11ec-8c2d-001a4aab830c -- bash -c 'ceph config get mgr mgr/cephadm/migration_current | grep 3'" Dead ================================================================= [6533700] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/mirror 3-final} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533700/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533700/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds [6533716] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533716/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533716/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds [6533734] orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533734/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533734/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds [6533752] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:00:00 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533752/ log: http://qa-proxy.ceph.com/teuthology/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533752/ Error reimaging machines: reached maximum tries (60) after waiting for 900 seconds Pass ================================================================= [6533674] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:10 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533674/ [6533676] orch:cephadm/orchestrator_cli/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 2-node-mgr agent/off orchestrator_cli} time: 00:28:01 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533676/ [6533677] orch:cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:02 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533677/ [6533678] orch:cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:16:30 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533678/ [6533679] orch:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:14:41 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533679/ [6533680] orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/basic 3-final} time: 00:12:36 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533680/ [6533681] orch:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:38:03 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533681/ [6533682] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:26:22 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533682/ [6533683] orch:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:36:30 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533683/ [6533684] orch:cephadm/workunits/{agent/off mon_election/classic task/test_adoption} time: 00:09:06 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533684/ [6533685] orch:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:24:57 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533685/ [6533686] orch:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:15:09 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533686/ [6533687] orch:cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:51:35 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533687/ [6533688] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:35:55 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533688/ [6533689] orch:cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:15:30 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533689/ [6533690] orch:cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:49 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533690/ [6533691] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:25:20 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533691/ [6533692] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:17 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533692/ [6533693] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/iscsi 3-final} time: 00:14:52 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533693/ [6533694] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async root} time: 00:31:24 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533694/ [6533695] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:47:08 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533695/ [6533696] orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_cephadm} time: 00:17:33 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533696/ [6533697] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:35:22 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533697/ [6533699] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:14:47 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533699/ [6533701] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v1only root} time: 00:37:51 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533701/ [6533702] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:38:57 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533702/ [6533704] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04-15.2.9 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:32:19 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533704/ [6533705] orch:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:25:09 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533705/ [6533706] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-bucket 3-final} time: 00:20:38 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533706/ [6533707] orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v2only root} time: 00:48:40 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533707/ [6533708] orch:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:36:43 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533708/ [6533709] orch:cephadm/workunits/{agent/off mon_election/classic task/test_cephadm_repos} time: 00:04:58 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533709/ [6533710] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:14:19 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533710/ [6533711] orch:cephadm/smoke/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:15:53 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533711/ [6533712] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:24:28 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533712/ [6533714] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/nfs-ingress-rgw-user 3-final} time: 00:20:22 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533714/ [6533715] orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async root} time: 01:00:53 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533715/ [6533717] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:33:08 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533717/ [6533718] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:36:09 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533718/ [6533719] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:17:31 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533719/ [6533720] orch:cephadm/smoke/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:19:30 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533720/ [6533721] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/nfs-ingress 3-final} time: 00:16:21 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533721/ [6533722] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v1only root} time: 00:36:07 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533722/ [6533723] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:47:01 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533723/ [6533724] orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_nfs} time: 00:27:54 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533724/ [6533725] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:25:29 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533725/ [6533726] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:36:44 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533726/ [6533727] orch:cephadm/orchestrator_cli/{0-random-distro$/{ubuntu_20.04} 2-node-mgr agent/on orchestrator_cli} time: 00:10:57 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533727/ [6533728] orch:cephadm/smoke-roleless/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs-ingress2 3-final} time: 00:16:15 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533728/ [6533729] orch:cephadm/smoke-singlehost/{0-random-distro$/{ubuntu_20.04} 1-start 2-services/rgw 3-final} time: 00:14:04 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533729/ [6533730] orch:cephadm/thrash/{0-distro/centos_8.2_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async-v2only root} time: 00:37:57 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533730/ [6533731] orch:cephadm/upgrade/{1-start-distro/1-start-ubuntu_20.04 2-repo_digest/defaut 3-start-upgrade 4-wait 5-upgrade-ls agent/off mon_election/classic} time: 00:31:19 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533731/ [6533732] orch:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:34:50 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533732/ [6533733] orch:cephadm/osds/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:18:31 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533733/ [6533735] orch:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:23:55 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533735/ [6533736] orch:cephadm/smoke-roleless/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-services/nfs 3-final} time: 00:14:30 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533736/ [6533737] orch:cephadm/thrash/{0-distro/centos_8.3_container_tools_3.0 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async root} time: 00:36:43 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533737/ [6533738] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:35:12 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533738/ [6533739] orch:cephadm/workunits/{agent/off mon_election/classic task/test_orch_cli} time: 00:13:28 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533739/ [6533740] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:22:51 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533740/ [6533741] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:29:13 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533741/ [6533742] orch:cephadm/osds/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-ops/rm-zap-flag} time: 00:13:54 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533742/ [6533743] orch:cephadm/smoke/{0-distro/ubuntu_20.04 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:17:30 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533743/ [6533744] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-services/nfs2 3-final} time: 00:13:47 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533744/ [6533745] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v1only root} time: 00:49:21 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533745/ [6533746] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:46:41 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533746/ [6533748] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:35:48 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533748/ [6533749] orch:cephadm/smoke-roleless/{0-distro/centos_8.stream_container_tools_crun 0-nvme-loop 1-start 2-services/rgw-ingress 3-final} time: 00:16:24 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533749/ [6533750] orch:cephadm/thrash/{0-distro/centos_8.stream_container_tools_crun 1-start 2-thrash 3-tasks/small-objects fixed-2 msgr/async-v2only root} time: 00:30:45 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533750/ [6533751] orch:cephadm/with-work/{0-distro/ubuntu_20.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:37:24 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533751/ [6533753] orch:cephadm/osds/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop 1-start 2-ops/rm-zap-wait} time: 00:13:20 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533753/ [6533754] orch:cephadm/smoke/{0-distro/centos_8.2_container_tools_3.0 0-nvme-loop agent/on fixed-2 mon_election/connectivity start} time: 00:16:04 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533754/ [6533755] orch:cephadm/upgrade/{1-start-distro/1-start-centos_8.stream_container-tools 2-repo_digest/repo_digest 3-start-upgrade 4-wait 5-upgrade-ls agent/on mon_election/connectivity} time: 00:26:46 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533755/ [6533756] orch:cephadm/with-work/{0-distro/centos_8.2_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v1only start tasks/rados_python} time: 00:23:47 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533756/ [6533757] orch:cephadm/workunits/{agent/on mon_election/connectivity task/test_adoption} time: 00:07:35 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533757/ [6533758] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_3.0 0-nvme-loop 1-start 2-services/rgw 3-final} time: 00:19:52 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533758/ [6533759] orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_3.0 1-start 2-thrash 3-tasks/snaps-few-objects fixed-2 msgr/async root} time: 00:48:17 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533759/ [6533760] orch:cephadm/with-work/{0-distro/centos_8.3_container_tools_3.0 fixed-2 mode/packaged mon_election/classic msgr/async-v2only start tasks/rados_api_tests} time: 00:34:57 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533760/ [6533761] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_python} time: 00:23:06 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533761/ [6533762] orch:cephadm/osds/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop 1-start 2-ops/repave-all} time: 00:13:34 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533762/ [6533763] orch:cephadm/smoke/{0-distro/centos_8.3_container_tools_3.0 0-nvme-loop agent/off fixed-2 mon_election/classic start} time: 00:15:29 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533763/ [6533764] orch:cephadm/smoke-roleless/{0-distro/rhel_8.4_container_tools_rhel8 0-nvme-loop 1-start 2-services/basic 3-final} time: 00:18:04 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533764/ [6533765] orch:cephadm/thrash/{0-distro/rhel_8.4_container_tools_rhel8 1-start 2-thrash 3-tasks/rados_api_tests fixed-2 msgr/async-v1only root} time: 00:47:28 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533765/ [6533766] orch:cephadm/with-work/{0-distro/centos_8.stream_container_tools_crun fixed-2 mode/packaged mon_election/classic msgr/async-v1only start tasks/rados_api_tests} time: 00:35:18 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533766/ [6533767] orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:32:09 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533767/ [6533768] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_3.0 fixed-2 mode/root mon_election/connectivity msgr/async-v2only start tasks/rados_python} time: 00:35:00 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533768/ [6533769] orch:cephadm/workunits/{agent/off mon_election/classic task/test_cephadm} time: 00:15:47 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533769/ [6533770] orch:cephadm/smoke-roleless/{0-distro/ubuntu_20.04 0-nvme-loop 1-start 2-services/client-keyring 3-final} time: 00:13:19 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533770/ [6533771] orch:cephadm/thrash/{0-distro/ubuntu_20.04 1-start 2-thrash 3-tasks/radosbench fixed-2 msgr/async-v2only root} time: 00:53:48 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533771/ [6533772] orch:cephadm/with-work/{0-distro/rhel_8.4_container_tools_rhel8 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} time: 00:48:54 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533772/ [6533773] orch:cephadm/osds/{0-distro/centos_8.stream_container_tools 0-nvme-loop 1-start 2-ops/rm-zap-add} time: 00:13:22 info: http://pulpito.ceph.com/swagner-2021-11-29_16:02:52-orch:cephadm-wip-swagner3-testing-2021-11-29-1155-distro-default-smithi/6533773/