2020-05-25T05:26:35.572 INFO:root:teuthology version: 1.0.0-4413d31 2020-05-25T05:26:35.574 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/kchai-2020-05-25_05:21:59-rgw-nautilus-distro-basic-smithi/5090803 --name kchai-2020-05-25_05:21:59-rgw-nautilus-distro-basic-smithi --block --verbose --description rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/filestore-xfs.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_roundtrip.yaml} --owner scheduled_kchai@teuthology --lock -- /tmp/teuthology-worker.48gqc0wg.tmp 2020-05-25T05:26:35.598 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-25T05:26:35.666 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/kchai-2020-05-25_05:21:59-rgw-nautilus-distro-basic-smithi/5090803 branch: nautilus description: rgw/thrash/{civetweb.yaml clusters/fixed-2.yaml install.yaml objectstore/filestore-xfs.yaml thrasher/default.yaml thrashosds-health.yaml workload/rgw_roundtrip.yaml} email: tchaikov@gmail.com first_in_suite: false job_id: '5090803' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: kchai-2020-05-25_05:21:59-rgw-nautilus-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 overrides: admin_socket: branch: nautilus ceph: conf: mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: debug filestore: 20 debug journal: 20 debug ms: 20 debug osd: 25 osd objectstore: filestore osd shutdown pgref assert: true osd sloppy crc: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request sha1: ccd9c04f88e53aef7e4f1068ce1221fa3b97450d ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: sha1: ccd9c04f88e53aef7e4f1068ce1221fa3b97450d rgw: frontend: civetweb workunit: branch: nautilus sha1: ccd9c04f88e53aef7e4f1068ce1221fa3b97450d owner: scheduled_kchai@teuthology priority: 90 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: ccd9c04f88e53aef7e4f1068ce1221fa3b97450d sleep_before_teardown: 0 suite: rgw suite_branch: nautilus suite_path: /home/teuthworker/src/github.com_ceph_ceph_nautilus/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: ccd9c04f88e53aef7e4f1068ce1221fa3b97450d tasks: - install: null - ceph: null - rgw: - client.0 - thrashosds: ceph_objectstore_tool: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 chance_test_min_size: 0 op_delay: 30 timeout: 1200 - s3roundtrip: client.0: force-branch: ceph-nautilus rgw_server: client.0 roundtrip: bucket: rttest duration: 300 files: num: 10 size: 2000 stddev: 500 readers: 10 writers: 3 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.8446 2020-05-25T05:26:35.667 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_nautilus/qa; will attempt to use it 2020-05-25T05:26:35.668 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_nautilus/qa/tasks 2020-05-25T05:26:35.668 INFO:teuthology.run_tasks:Running task internal.check_packages... 2020-05-25T05:26:35.672 INFO:teuthology.task.internal:Checking packages... 2020-05-25T05:26:35.673 INFO:teuthology.task.internal:Checking packages skipped, missing os_type 'None' or ceph hash 'ccd9c04f88e53aef7e4f1068ce1221fa3b97450d' 2020-05-25T05:26:35.673 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2020-05-25T05:26:35.694 INFO:teuthology.task.internal:no buildpackages task found 2020-05-25T05:26:35.694 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2020-05-25T05:26:35.711 INFO:teuthology.task.internal.lock_machines:Locking machines... 2020-05-25T05:26:35.712 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2020-05-25T05:26:36.023 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:26:46.042 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:26:56.058 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:27:06.074 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:27:16.092 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-25T05:27:26.109 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-25T05:27:36.130 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-25T05:27:46.149 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-25T05:27:56.169 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-25T05:28:06.190 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-25T05:28:16.209 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 0)... 2020-05-25T05:28:26.228 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:28:36.249 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:28:46.268 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:28:56.288 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:29:06.309 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:29:16.327 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:29:26.344 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:29:36.365 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:29:46.384 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:29:56.408 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)... 2020-05-25T05:30:06.424 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 0 + 2, have 1)...