2022-01-29T02:16:54.595 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-29T02:16:54.597 INFO:teuthology.results:subset: '111/120000' 2022-01-29T02:16:54.597 INFO:teuthology.results:seed: '6102' 2022-01-29T07:29:12.212 INFO:root:teuthology version: 1.1.0-7c0cb867 2022-01-29T07:29:12.260 INFO:teuthology.results:Waiting up to 43200 seconds for tests to finish... 2022-01-29T12:10:34.660 INFO:teuthology.results:Tests finished! gathering results... 2022-01-29T12:10:46.434 INFO:scrape:Found 99 jobs 2022-01-29T12:10:46.564 WARNING:scrape:Not-understood crash indication 2022-01-29T03:07:04.586 INFO:tasks.rgw.c2.client.0.smithi192.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:46.565 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:46.583 WARNING:scrape:Not-understood crash indication 2022-01-29T03:07:04.586 INFO:tasks.rgw.c2.client.0.smithi192.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:46.585 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:46.602 WARNING:scrape:Not-understood crash indication 2022-01-29T03:07:04.586 INFO:tasks.rgw.c2.client.0.smithi192.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:46.603 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:46.621 WARNING:scrape:Not-understood crash indication 2022-01-29T03:07:04.586 INFO:tasks.rgw.c2.client.0.smithi192.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:46.622 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:46.827 WARNING:scrape:Not-understood crash indication 2022-01-29T05:57:34.295 INFO:tasks.rgw.c2.client.0.smithi160.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:46.828 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:46.847 WARNING:scrape:Not-understood crash indication 2022-01-29T05:57:34.295 INFO:tasks.rgw.c2.client.0.smithi160.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:46.848 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:46.992 WARNING:scrape:Not-understood crash indication 2022-01-29T07:09:33.094 INFO:tasks.rgw.c2.client.0.smithi105.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:46.993 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:47.013 WARNING:scrape:Not-understood crash indication 2022-01-29T07:09:33.094 INFO:tasks.rgw.c2.client.0.smithi105.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:47.014 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:47.154 WARNING:scrape:Not-understood crash indication 2022-01-29T04:27:18.594 INFO:tasks.rgw.c2.client.0.smithi089.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:47.155 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:47.173 WARNING:scrape:Not-understood crash indication 2022-01-29T04:27:18.594 INFO:tasks.rgw.c2.client.0.smithi089.stderr:daemon-helper: command crashed with signal 11 2022-01-29T12:10:47.174 WARNING:scrape:Not-understood crash indication 2022-01-29T12:10:47.352 INFO:scrape:Found 3 distinct failure reasons 2022-01-29T12:10:47.353 INFO:scrape:Failure: Command failed on smithi175 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' 2022-01-29T12:10:47.354 INFO:scrape:8 jobs: ['6647198', '6647249', '6647147', '6647284', '6647170', '6647294', '6647270', '6647221'] 2022-01-29T12:10:47.354 INFO:scrape:suites intersection: ['1-ceph-install/octopus', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-quincy', '8-final-workload/{rbd-python', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack'] 2022-01-29T12:10:47.355 INFO:scrape:suites union: ['1-ceph-install/octopus', '1.1-pg-log-overrides/normal_pg_log', '1.1-pg-log-overrides/short_pg_log', '2-partial-upgrade/firsthalf', '3-thrash/default', '4-workload/{radosbench', '5-finish-upgrade', '6-quincy', '8-final-workload/{rbd-python', 'mon_election/classic', 'mon_election/connectivity', 'objectstore/bluestore-hybrid', 'objectstore/filestore-xfs', 'rbd-cls', 'rbd-import-export', 'rbd_api', 'readwrite', 'rgw_ragweed_prepare', 'snaps-few-objects}', 'snaps-many-objects}', 'start}', 'thrashosds-health', 'ubuntu_20.04}', 'upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack'] 2022-01-29T12:10:47.355 INFO:scrape: 2022-01-29T12:10:47.356 INFO:scrape:Failure: rgw multisite test failures 2022-01-29T12:10:47.356 INFO:scrape:4 jobs: ['6647141', '6647241', '6647280', '6647192'] 2022-01-29T12:10:47.357 INFO:scrape:suites intersection: ['frontend', 'overrides', 'realm', 'tasks', 'upgrade/octopus-x/rgw-multisite/{clusters'] 2022-01-29T12:10:47.357 INFO:scrape:suites union: ['frontend', 'overrides', 'realm', 'tasks', 'upgrade/octopus-x/rgw-multisite/{clusters', 'upgrade/primary}', 'upgrade/secondary}'] 2022-01-29T12:10:47.358 INFO:scrape: 2022-01-29T12:10:47.358 INFO:scrape:Failure: Command failed on smithi140 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3397fcc0-80d1-11ec-8c35-001a4aab830c -- ceph orch daemon add osd smithi140:vg_nvme/lv_4' 2022-01-29T12:10:47.359 INFO:scrape:1 jobs: ['6647283'] 2022-01-29T12:10:47.359 INFO:scrape:suites: ['1-ranks/2', '1-volume/{0-create', '2-allow_standby_replay/no', '2-client', '3-upgrade-with-workload', '3-verify}', '4-verify}}', 'centos_8.stream_container_tools', 'conf/{client', 'mds', 'mon', 'osd}', 'overrides/{pg-warn', 'roles', 'syntax', 'tasks/{0-from/v16.2.4', 'upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap', 'whitelist_health', 'whitelist_wrongly_marked_down}'] 2022-01-29T12:10:47.360 INFO:scrape: 2022-01-29T12:10:47.366 INFO:teuthology.results:Sending results to ceph-qa@ceph.io: Test Run: yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi ================================================================= info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/ logs: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/ failed: 13 dead: 0 running: 0 waiting: 0 queued: 0 passed: 86 Fail ================================================================= [6647141] upgrade/octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/primary} ----------------------------------------------------------------- time: 03:06:40 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647141/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647141/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c32074ccc5d34f6e912890d73876f5f2 rgw multisite test failures [6647147] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-hybrid thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 03:15:43 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647147/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647147/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e22a0bef2b4b4657aaa0318c3f2a4756 Command failed on smithi097 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6647170] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 01:58:38 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647170/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647170/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e0d1850eeec047a9830f8faae5da31c9 Command failed on smithi195 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6647192] upgrade/octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary} ----------------------------------------------------------------- time: 03:07:02 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647192/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647192/ sentry: https://sentry.ceph.com/organizations/ceph/?query=32d1e91841d04ba68fdc8f8088c414a8 rgw multisite test failures [6647198] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-hybrid thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 03:35:16 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647198/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647198/ sentry: https://sentry.ceph.com/organizations/ceph/?query=7d3f61786afc4679a2e51c0d2757bb9c Command failed on smithi175 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6647221] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 01:44:11 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647221/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647221/ sentry: https://sentry.ceph.com/organizations/ceph/?query=383dad2ec38243b9aef875429d18c8e3 Command failed on smithi170 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6647241] upgrade/octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/primary} ----------------------------------------------------------------- time: 03:06:24 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647241/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647241/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e8e5ad3b136a4d96a51b34ecc86ebf5b rgw multisite test failures [6647249] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/bluestore-hybrid thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 03:42:35 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647249/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647249/ sentry: https://sentry.ceph.com/organizations/ceph/?query=94901e0888ae46efb4ab0ca5c8a6764e Command failed on smithi204 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6647270] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/filestore-xfs thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 01:49:31 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647270/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647270/ sentry: https://sentry.ceph.com/organizations/ceph/?query=15fd0cc985234dae9c45698fabc23580 Command failed on smithi152 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6647280] upgrade/octopus-x/rgw-multisite/{clusters frontend overrides realm tasks upgrade/secondary} ----------------------------------------------------------------- time: 03:07:43 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647280/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647280/ sentry: https://sentry.ceph.com/organizations/ceph/?query=c86784cf3be54ed6aed26bcdad418f0c rgw multisite test failures [6647283] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} ----------------------------------------------------------------- time: 00:11:39 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647283/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647283/ sentry: https://sentry.ceph.com/organizations/ceph/?query=374a8cf1bfec49daa2bfbabeeee1e0f4 Command failed on smithi140 with status 22: 'sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3397fcc0-80d1-11ec-8c35-001a4aab830c -- ceph orch daemon add osd smithi140:vg_nvme/lv_4' [6647284] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/connectivity objectstore/bluestore-hybrid thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 03:32:44 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647284/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647284/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e9a84d5f0f7d421dae12d060f80e4211 Command failed on smithi170 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' [6647294] upgrade/octopus-x/stress-split-no-cephadm/{0-cluster/{openstack start} 1-ceph-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 4-workload/{radosbench rbd-cls rbd-import-export rbd_api readwrite rgw_ragweed_prepare snaps-few-objects} 5-finish-upgrade 6-quincy 8-final-workload/{rbd-python snaps-many-objects} mon_election/classic objectstore/filestore-xfs thrashosds-health ubuntu_20.04} ----------------------------------------------------------------- time: 01:55:13 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647294/ log: http://qa-proxy.ceph.com/teuthology/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647294/ sentry: https://sentry.ceph.com/organizations/ceph/?query=e3a7712a4dc548268581e683130eb0a7 Command failed on smithi137 with status 2: 'cd /home/ubuntu/cephtest/ragweed && ./bootstrap' Pass ================================================================= [6647133] upgrade/pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:24:09 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647133/ [6647135] upgrade/octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:11:56 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647135/ [6647137] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:25:55 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647137/ [6647139] upgrade/octopus-x/parallel/{0-distro/centos_8.2_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:04:26 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647139/ [6647143] upgrade/pacific-x/parallel/{0-random-distro$/{rhel_8.4_container_tools_rhel8} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:30:14 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647143/ [6647145] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/classic thrashosds-health ubuntu_20.04} time: 04:35:14 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647145/ [6647149] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:30 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647149/ [6647151] upgrade/octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:05:21 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647151/ [6647153] upgrade/pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:25:06 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647153/ [6647155] upgrade/octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:17:05 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647155/ [6647157] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:54 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647157/ [6647159] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:43 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647159/ [6647161] upgrade/cephfs/nofs/{bluestore-bitmap centos_latest conf/{client mds mon osd} no-mds-cluster overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-upgrade}} time: 00:10:28 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647161/ [6647163] upgrade/octopus-x/stress-split/{0-distro/centos_8.2_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:50:01 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647163/ [6647165] upgrade/pacific-x/stress-split/{0-distro/centos_8.2_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:50:11 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647165/ [6647167] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/connectivity thrashosds-health ubuntu_20.04} time: 01:38:46 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647167/ [6647172] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:15:49 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647172/ [6647174] upgrade/pacific-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:02:09 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647174/ [6647176] upgrade/octopus-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:09:59 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647176/ [6647178] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:31:22 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647178/ [6647180] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:16 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647180/ [6647182] upgrade/octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:08:02 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647182/ [6647184] upgrade/pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:55:44 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647184/ [6647186] upgrade/octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:49:16 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647186/ [6647188] upgrade/octopus-x/parallel/{0-distro/centos_8.2_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:06:05 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647188/ [6647190] upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:19:24 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647190/ [6647194] upgrade/pacific-x/parallel/{0-random-distro$/{ubuntu_20.04} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:23:33 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647194/ [6647196] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/classic thrashosds-health ubuntu_20.04} time: 02:12:17 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647196/ [6647200] upgrade/octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:04:55 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647200/ [6647202] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:27 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647202/ [6647204] upgrade/pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:10:04 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647204/ [6647206] upgrade/octopus-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:09:14 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647206/ [6647208] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:00 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647208/ [6647210] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:14 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647210/ [6647212] upgrade/octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:24:40 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647212/ [6647214] upgrade/pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:22:55 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647214/ [6647216] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:23:23 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647216/ [6647218] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/connectivity thrashosds-health ubuntu_20.04} time: 03:57:38 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647218/ [6647223] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:15:50 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647223/ [6647225] upgrade/pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:14:07 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647225/ [6647227] upgrade/octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:21:53 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647227/ [6647229] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:17:20 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647229/ [6647231] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:44 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647231/ [6647233] upgrade/octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:09:08 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647233/ [6647235] upgrade/pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:18:11 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647235/ [6647237] upgrade/octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:58:02 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647237/ [6647239] upgrade/octopus-x/parallel/{0-distro/centos_8.2_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:05:35 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647239/ [6647243] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/no}} time: 00:26:02 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647243/ [6647245] upgrade/pacific-x/parallel/{0-random-distro$/{centos_8.stream_container_tools} 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:15:33 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647245/ [6647247] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/classic thrashosds-health ubuntu_20.04} time: 04:11:42 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647247/ [6647251] upgrade/octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:03:34 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647251/ [6647253] upgrade/pacific-x/stress-split/{0-distro/centos_8.2_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:08:51 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647253/ [6647255] upgrade/octopus-x/stress-split/{0-distro/centos_8.2_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:26:30 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647255/ [6647257] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/v16.2.4 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:30:26 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647257/ [6647259] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:24 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647259/ [6647261] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:08 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647261/ [6647263] upgrade/octopus-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:49:35 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647263/ [6647265] upgrade/pacific-x/stress-split/{0-distro/centos_8.3_container_tools_3.0 0-roles 1-start 2-first-half-tasks/readwrite 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:54:30 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647265/ [6647267] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/connectivity thrashosds-health ubuntu_20.04} time: 02:12:26 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647267/ [6647269] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:26:50 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647269/ [6647271] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:17:10 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647271/ [6647272] upgrade/pacific-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:22:06 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647272/ [6647273] upgrade/octopus-x/stress-split/{0-distro/centos_8.stream_container_tools 0-roles 1-start 2-first-half-tasks/snaps-few-objects 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:14:47 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647273/ [6647274] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:15:48 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647274/ [6647275] upgrade/octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:11:34 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647275/ [6647276] upgrade/cephfs/featureful_client/upgraded_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/yes pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-client-upgrade 4-compat_client 5-client-sanity}} time: 00:19:07 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647276/ [6647277] upgrade/pacific-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:59:52 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647277/ [6647278] upgrade/octopus-x/stress-split/{0-distro/centos_8.stream_container_tools_crun 0-roles 1-start 2-first-half-tasks/radosbench 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 01:56:47 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647278/ [6647279] upgrade/octopus-x/parallel/{0-distro/centos_8.2_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:05:33 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647279/ [6647281] upgrade/pacific-x/parallel/{0-random-distro$/{centos_8.2_container_tools_3.0} 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api test_rbd_python}} time: 01:18:11 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647281/ [6647282] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/short_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/filestore-xfs 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/classic thrashosds-health ubuntu_20.04} time: 01:39:59 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647282/ [6647285] upgrade/octopus-x/parallel/{0-distro/centos_8.3_container_tools_3.0 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:04:28 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647285/ [6647286] upgrade/pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:20:20 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647286/ [6647287] upgrade/octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_3.0 0-roles 1-start 2-first-half-tasks/rbd-cls 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:18:13 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647287/ [6647288] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:02:28 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647288/ [6647289] upgrade/cephfs/featureful_client/old_client/{bluestore-bitmap centos_latest clusters/1-mds-2-client-micro conf/{client mds mon osd} overrides/{multimds/no pg-warn whitelist_health whitelist_wrongly_marked_down} tasks/{0-octopus 1-client 2-upgrade 3-compat_client/pacific}} time: 00:23:05 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647289/ [6647290] upgrade/octopus-x/parallel/{0-distro/centos_8.stream_container_tools_crun 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:01:35 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647290/ [6647291] upgrade/octopus-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:28:43 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647291/ [6647292] upgrade/pacific-x/stress-split/{0-distro/rhel_8.4_container_tools_rhel8 0-roles 1-start 2-first-half-tasks/rbd-import-export 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/rbd-import-export mon_election/classic} time: 02:33:33 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647292/ [6647293] upgrade/octopus-x/stress-split-erasure-code-no-cephadm/{0-cluster/{openstack start} 1-octopus-install/octopus 1.1-pg-log-overrides/normal_pg_log 2-partial-upgrade/firsthalf 3-thrash/default 3.1-objectstore/bluestore-hybrid 4-ec-workload/{rados-ec-workload rbd-ec-workload} 5-finish-upgrade 6-quincy 7-final-workload mon_election/connectivity thrashosds-health ubuntu_20.04} time: 04:54:00 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647293/ [6647295] upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_8.stream_container_tools conf/{client mds mon osd} overrides/{pg-warn syntax whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-from/pacific 1-volume/{0-create 1-ranks/1 2-allow_standby_replay/yes 3-verify} 2-client 3-upgrade-with-workload 4-verify}} time: 00:27:50 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647295/ [6647296] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_3.0 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:22 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647296/ [6647297] upgrade/pacific-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:28:39 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647297/ [6647298] upgrade/octopus-x/stress-split/{0-distro/ubuntu_20.04 0-roles 1-start 2-first-half-tasks/rbd_api 3-stress-tasks/{radosbench rbd-cls rbd-import-export rbd_api readwrite snaps-few-objects} 4-second-half-tasks/radosbench mon_election/connectivity} time: 02:25:14 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647298/ [6647299] upgrade/octopus-x/parallel/{0-distro/rhel_8.4_container_tools_rhel8 0-start 1-tasks mon_election/classic upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:16:52 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647299/ [6647300] upgrade/octopus-x/parallel/{0-distro/ubuntu_20.04 0-start 1-tasks mon_election/connectivity upgrade-sequence workload/{ec-rados-default rados_api rados_loadgenbig rbd_import_export test_rbd_api}} time: 01:09:16 info: http://pulpito.ceph.com/yuriw-2022-01-28_20:43:34-upgrade-wip-yuriw-quincy-1.28.22-distro-default-smithi/6647300/